Warning: this blog is an experiment. The idea for a blog came from a real conversation I had with somebody. The content is researched, debated, refined, “hardened” by a Gemini skill I created (using Gemini 4 Argon). Then Claude Opus 5.5 performed a peer review, and tore some parts to shreds. Then rewrote them. And then I very lightly edited it, and tuned a few jokes. I replaced a food fight joke with a pillow fight joke, specifically. Let me know if this sucks, and I will stop. Promise!

—

A security peer recently shared a proposal they had just heard in the wild: instead of paying for a SIEM, why not load several days of raw log data into a locally run open-weights model, ask it “what looks risky?”, and review whatever the model highlights?

No SIEM ingest tax! No cloud token bills! No SIEM license fees! No writing correlation rules! No tuning anomaly detection! Just a gaming GPU under a desk and pure, artisanal AI-native vibes. Ahhh, magic!

Wait. Really?

My first instinct, forged by 24 years of staring at logs since I first met a SIM in January 2002, was to laugh, pour a drink, and move on. Surely everyone knows this is a terrible idea, right? Right?!

Wrong.

If an engineer, an IT director, or a cost-cutting CFO pitches “local LLM as a poor man’s SIEM” in your organization, you cannot just roll your eyes and mutter “hallucinations.” People keep inventing this idea for real reasons. SIEM ingest pricing still hurts: some vendor cost explainer works out 5 TB per day at $2 to $4 per GB as $3.6 to $7.3 million a year on ingestion alone. Detection engineering is still hard work. And yes the AI hype machine has spent two years promising executives that models can “reason over any unstructured data.”

So let’s dissect why dumping a week of raw logs into a local model breaks four basic laws of “operational physics”, and then carve out the narrow 10% where a local model does help with logs. The short version fits on a sticker: the LLM writes the parser, it never runs as the parser. And the twist is that the problem here is “raw,” not “local.”

Token Arithmetic and Naive Chunking (Or: How Big Do You Think 7 Days of Logs Is?!)

Let’s start with grade-school arithmetic, because clearly nobody pitching this architecture has ever opened a calculator…

Forget the enterprise. Take a tiny environment, a single server rack generating a measly 1 GB of raw text logs per day. Seven days of logs is 7 GB of text, or roughly 1.75 billion tokens at 4 characters per token. That is generous, since IP addresses, GUIDs and hex strings tokenize worse than English (Wow, really? Yes, really — at least Claude thinks so! He would know, right?)

Now look at that “free” 8B or 70B parameter model on your GPU workstation. Its usable context window, before the KV cache devours your VRAM and the fans sound like a Boeing 747 on takeoff, is maybe 32K to 128K tokens. You are shoving 1.75 billion tokens through a 128K keyhole, which is about 13,700 full context windows. Painful enough yet?

Well, it gets worse. Assume your box reads a generous 2,000 tokens per second (benchmark your own, I’ll wait). Reading those 1.75 billion tokens takes about 10 days. You need 10 days to read 7 days of logs, so you never catch up! Scale that to the 5 TB per day example above and it is about 20 years of GPU time for each day of logs. Hilarity ensues! Replacing a deterministic stream processor with a token-by-token text predictor is like replacing your municipal water pipes with a guy carrying a silver espresso spoon from a well 10 miles away.

“Fine, Anton,” the DIY enthusiast says, “we’ll chunk the logs into 2,000-line batches and loop the local model over them all weekend!” Or we will get a bigger boat… eh… server rack to run the model.

Chunking naively, by just loading the data in batches, fails hilariously. Here is how:

• Attack chains get cut. The phish on Tuesday lands in one batch, the lateral movement on Thursday lands thousands of batches later, and no prompt ever sees both. Each batch gets judged alone.
• Merging the summaries does not bring the signal back. By the time you summarize the summaries, the one quiet line was already dropped in round one. So “naive+” fails as well.
• Position inside the batch matters. Liu et al.’s “Lost in the Middle” found that models do best when the relevant information sits at the start or end of a long context and degrade sharply in the middle. They tested question answering and key-value retrieval, not logs, but picture a quiet privilege escalation on line 1,482 of your batch. Worst seat in the house.

Researchers keep running into this. Cai, Ryan and Karie note that iterative approach is limited by context capacity and that map-reduce style summarization is computationally heavy (and requires some luck). Even so, that is a proof of concept on up to 20,000 rows per dataset, and they say enterprise volumes remain to be validated.

So smarter chunking exists (reminder: it is not magic, in this area nothing truly is): parse, cluster, then retrieve by entity and time. At that point you are building the first half of a SIEM. Congratulations!

As I wrote in The Ancient Art of SIEM, cost pain is eternal and only the currency changes. In 2003 you ran out of hardware, in 2015 you ran out of cloud budget, and in 2026 you run out of tokens and VRAM.

The Input-Centric Time Machine: Back to 2002 Batch Review

Notice the most revealing phrase in this idea: “load 1 to 7 days of log data… and then review what the model highlights.”

That is not real-time detection. That is batch-mode log archeology! When intruders can go from initial access to data theft in hours (soon: minutes for some incidents), reviewing a week of logs in a batch means you are running a weekly book club about how you got breached last Wednesday. But hey … is your SIEM faster? Well, it should be!

Worse, asking an LLM to “highlight what looks risky” violates the most important lesson of two decades of detection engineering. In Output-driven SIEM, 13 years later, I revisited my favorite 2011 mental model: think about what you want to see at the output before you collect, and AI does not change this. Asking a model to “find risk” in raw logs with zero detection engineering is the ultimate input-centric anti-pattern. It failed with ML in mid-2010s, and LLM is not different (in this regard).

Now, about determinism. The lazy objection is “the answers change on every run,” and a sharp engineer will reply “so pin the seed.” Fair enough! Run-to-run randomness is an engineering problem: Thinking Machines got 80 unique completions out of 1,000 at temperature 0, then 1,000 identical ones after fixing their kernels. But is it actually bad? Maybe if you run 5 times, you will find 2x more “fun” things. But hey … also probably at least 5x more noise.

The harder test is stability. Take the same 48 hours of domain controller logs, shuffle the order or change one irrelevant line, and run it again. Given what we learned, I would bet real money the verdict moves. Well-crafted SIEM rule does not care what order the lines arrive in.

Zero Local Context Equals Hallucinated Risk

Why can’t a local model tell an active nation-state intrusion from Tuesday’s messy IT administration? Because raw logs do not contain your organizational reality! Shocking, eh?

A raw syslog or CloudTrail line says that svc_deploy_04 logged into prod-db-02 at 3:14 AM from an internal jump host and ran an encoded PowerShell script. Is that a five-alarm breach or the legacy billing batch job that Bob in accounting wrote in 2017 and nobody dares to touch? This is a dated example, on purpose. Insert your own fun example involving a shadow AI agent!

In Part 3 of the Stop Building a 2003 SOC series, I argued that the #1 reason AI fails in the SOC is the absence of Context as Code: machine-readable, owned, and expiring knowledge of your assets, business processes, and weird administrative exceptions. In Part 2, we warned about automation bias: humans reviewing plausible, well-written machine verdicts approve them.

Also, the test from Can AI Let You Jump SOC Maturity Levels? applies here: does this exist because someone had to spend time, or because someone had to learn something about us? Writing a parser is a time cost. Knowing about Bob’s job is a learning cost, and no model downloaded from the internet has paid it.

Feed raw logs into a naked local LLM with no CMDB enrichment, no threat intel, no identity graph, and no 90-day baseline, and its only source of “context” is its pre-trained weights. So it flags every scary-looking word (ERROR, DENIED, admin, root, failed) and writes you a convincing fictional thriller about your own network.

The Confused Deputy Trap: When the Attacker Prompts Your “SIEM” via User-Agent

Now put on an attacker hat for 30 seconds. What is a log file?

A log file is untrusted, attacker-influenced input. Every public-facing web server, SSH daemon, API gateway, and email relay faithfully records whatever string an outsider sends in an HTTP User-Agent, a failed username field, or a JSON payload.

Log processors have been attacked through logged strings before (hi, Log4Shell!), but that was a bug you could patch. In a raw-log-to-LLM setup there is no parser boundary to patch, because instructions and data share one channel. You just handed the attacker write access to your detection engine’s prompt!

Summaries are also scary, BTW. Pandey and Bhujang found that the classic “ignore all instructions” override failed completely against the one model they tested, but context manipulation reached 96% success on summarization with no defenses and 38% even with constrained output.

In How to Solve the Mystery of AI Defense in Depth (and my October 2026 collaboration with Usman Chaudhary, The Model Has Preferences, but the Harness Got Controls), I emphasized that a model’s statistical preferences are never a security boundary, and only a deterministic harness is. Feed unparsed logs straight into a local LLM with no harness, and any kid with curl can blind your review or flood your team with fabricated critical alerts…

The Problem Is “Raw,” Not “Local”

Let’s be fair to the local crowd, because they have a point.

• Local solves a real problem. Logs are full of secrets, PII and internal hostnames. Not shipping them to a third-party API is a legitimate reason to want a model on your own hardware. It also — theoretically — eliminates an endless stream of unpredictable token costs flowing to a frontier model provider.
• Small models are not the villain. In the same paper, a structured design called Aida (separate agents for investigating, challenging and judging, working through a SIEM) used open-weight Qwen3.5 9B and it was workable for the task they had.
• “What about an agent querying a log store?” Now we are talking! The model forms a question, a database searches 7 days of logs deterministically, and the model reads a small result. That is a very different architecture, and it needs parsed data, a query engine, detections to start from, retention and access control. Sound familiar?

So keep your local model if you want it. What is missing from this proposal is the SIEM…

The Raw-Logs-to-Local-LLM Delusion Matrix

For explaining this to an enthusiastic colleague (or a budget-slashing executive) without starting a pillow fight:

Wait, Is a Local LLM Good for Anything in Log Land? (The 10% Where It Shines!)

I am a constructive cynic, not a Luddite (At least Gemini 4 Argon thinks so!). As I noted in the SOC maturity post, AI can jump the artifacts, and parsers are on that list (with caveats for tricky log sources). So point your local model at logs as an engineering sidecar, not an ingestion engine:

• Writing the parser, not being the parser. Instead of feeding 50 million lines from a weird legacy mainframe into an LLM, feed 20 sample lines and ask for a regex, OCSF mapping, or pipeline rule. The LLM writes the code once, and deterministic CPU code parses the 50 million lines at wire speed. This mostly works, I swear.
• Post-trigger micro-slice investigation. Once a rule or EDR alert fires on a specific host and 15-minute window, pull 200 pre-filtered, enriched log lines and ask the model for a timeline. Those lines still contain attacker strings, so treat the summary as untrusted and let the model request actions only from a closed list.
• Ad-hoc query translation. Translate an analyst’s question into KQL, SPL, or SQL, so the database searches 7 days of logs deterministically. This also mostly works, but some YMMV.

The Lean Team Reality

What if you are the CISO of a regional insurer or hospital with a small security team, a hybrid stack, strict state regulators, and an MDR provider? When an engineer or cost-conscious VP suggests “let’s dump our noisy firewall logs into a local Ollama box,” they aren’t trying to sabotage your SOC. They have a log source that is too expensive to send to the SIEM, and they want a free way to “keep an eye on it.” Here is how to solve that without a science project:

• Don’t use a GPU box as a poor man’s data lake. Route a noisy source to cheap, searchable object storage, or strip verbose fields in a pipeline before ingest. This is my old two-tier answer (a SIEM plus a broad, cheap log store), and it has aged well.
• Hold your MDR to the same standard. When they brag at the next QBR about “AI-powered log analysis,” ask whether deterministic detections run first or whether it is an LLM wrapper over unparsed telemetry. This point alone is worth the price of reading this blog (A.C. — this is me, not AI)
• Channel the DIY energy. Give your smart engineer 50 anonymized lines from your gnarliest legacy application and ask for three high-signal filter rules you can deploy in your real pipeline.

Action Items for Security Leaders

Is there anything you need to actually do?

• Run a napkin token audit on any AI log proposal (Monday morning). Before the demo starts, ask: “How many GB per day is the raw source, how fast does the model read, and what deterministic step selects the tiny slice that reaches a context window?” If the answer is “we’ll just load it in batches,” kill the project on the spot. It’s this “just” word.
• Fix ingest pain with tiering (30-day play). If this idea surfaced because a log source is too expensive for your SIEM, solve that: drop repetitive keepalive chatter, route compliance-only logs to cheap storage, and keep output-driven security data in your detection tier.
• Make “the LLM writes the parser, never runs as the parser” a standard (architecture play) — that is, if you write parsers at all. Allow LLMs at design time (parsers, mappings, queries from small samples) and at post-alert time (small, bounded, enriched slices), but not as the runtime ingestion or anomaly engine over raw logs. Put it in your MDR requirements too… otherwise they will keep whining that it takes 7 weeks to integrate a log source forever.
• Test your existing AI log workflows for prompt injection (60-day play). If your SOC or MDR summarizes anything containing attacker-controlled strings, test it in your next purple-team exercise. Do not stop at a naive “ignore instructions” canary, because passing that proves little. Try persona and context-rewriting payloads, plus payloads split across lines.

So, Do You Need a SIEM?

Call it a SIEM or call it a pile of parsers, a query engine, detections and retention. You need those things, and a GPU under a desk does not replace them. By all means run a model next to them, I promise it will be fun!

The part that worries me most is how this fails. A detection rule that misses can be tested, tuned, and re-run. A model that misses hands you a nice summary, and nobody knows what was not in it. As I said in September, skipped maturity used to fail loudly, and now it fails quietly.

We had 23 years of progress, and the best practice is still “have a plan and don’t ingest garbage.” That goes for your context window too.

[A.C. — note to self: this is v6 skill]

“I Don’t Need a SIEM, I Have Ollama!” (Yes, You Do!) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

Back in my analyst days, I built maturity models for a SOC (2018), a SIEM (2018), vulnerability management (2017) and threat intel (201?). Later, just for fun, I cooked up a simple SOAR adoption maturity model (2022). All of them were vaguely CMM-shaped: you start ad hoc, you get defined, you get measured, you get optimizing, magic! The ordering was not decorative. Each level existed because the one below it produced something the next level needed. Largely, you cannot jump levels, and if you do jump, you land in clown realm.

2026, now the pitch has changed. With AI (agentic, of course!), the story goes, a low maturity SOC can leap to maturity Level 3 or 4 “because AI”. No slog through the “defined process” desert, no “organic” growth through stages. Buy the agents, skip the years!

If you attended my RSA 2026 peer session, you saw the buyer-side version of this: “AI in a SOC sounds great, we will just wait for our SIEM/SOAR vendor or MDR to give it to us.” Same assumption, that the new version of the tool carries the maturity boost with it. Sadly, in reality AI can generate the paperwork of a mature SOC in an afternoon. It cannot give you the institutional memory to know what your own systems actually do.

Ten years ago, in “Jumping Security Maturity Fail”, I endorsed a reader’s line that you can jump technologies, but you can’t jump maturity. I’d like to re-examine that in the age of agents. Because some of it has changed. Just not the part vendors are selling. Think of this as “Jumping Security Maturity Fail, Part 2. 10 Years Later.”

Where this post sits

This is a bridge between two things I’ve been writing about. On one side, the AI-ready SOC pillars and how to plan the journey to them. On the other, the “Stop Building a 2003 SOC with AI” series (Part 2, Part 3), which argues the target SOC in 2026 should look nothing like the 2003 one (more on “SOC 2026” from scratch in a few days…)

The question in between: if the destination has changed, does the path still have mandatory stops? That is what “jumping maturity” is really asking.

The claim, restated so it can be tested

“AI lets you skip maturity stages” quietly conflates two different things:

1. Stage outputs — the artifacts a mature SOC has: playbooks, parsers, detections, runbooks, dashboards, workflows for various humans, skill profiles for humans to hire, etc
2. Stage prerequisites — the organizational state a mature SOC is in: it knows its environment, owns its processes, trusts its data, has a working feedback loop.

AI can generate the outputs of Level 3 for a Level 1 SOC in an afternoon. You can always say “add automation”, “make this playbook better faster”, “more AI…add even more AI…MOAR AAAIII!!!”

Still, it cannot generate the organizational prerequisites, because those are not documents. They are things an organization has learned about itself by operating. Once you split the journey along that line, the “jump” question mostly answers itself.

Typical SOC Maturity Journey (Woefully Oversimplified)

• Level 1 — Ad hoc. Some logs, some alerts, heroics. “Nobody knows what server4 does, but Joanna might, let’s call her.”
• Level 2 — Defined. SIEM with intentional use cases (hopefully, output-driven, not “what data do we have? Let’s shove it in!”), documented triage and (some) IR runbooks, named owners (for some things, some are even the right owners…), basic metrics.
• Level 3 — Managed. SOAR playbooks beyond “baby’s first phishing playbook”, some threat-informed detection, log source health monitoring (logs volume drop? alert!), MTTx that is tracked and acted on (sometimes).
• Level 4–5 — Optimizing. Detection engineering as a discipline (the DE series), a continuous detection / continuous response loop, hunting that feeds detection, an improvement culture. This sets you on a path to ASO end state.

So, let’s get “an AI jumprope” and start jumping!

Part A: What AI genuinely lets you jump

These are the labor-intensive artifacts of each level — the things that took quarters to write, not the things that took years to learn.

• Runbook and playbook authoring. Fifty credible playbooks, one afternoon. The writing effort that used to gate Level 2→3 is gone. This one you can jump. There are caveats, like that some of the playbooks will suck, but for many teams fixing a 80% playbook is not 20% faster than making one from scratch, but essentially 5X faster… SOC process docs, playbooks, etc is where jumping is legit.
• Parsers and normalization. Onboarding long-tail log sources was a classic Level 2 grind. Largely shortcut-able now (with lots of caveats; for tricky log sources the “fully machined” parser will suck). You can jump here, but you may trip and fall.
• Baseline detection content. “Reasonable coverage of common threats that matter to you” no longer requires a detection engineering team to bootstrap. You can “machine” some custom detection quick, and some will even work. Keep in mind, here you can jump but not very far: you won’t arrive on “full auto” process for turning intel into detections that work well for you.
• Tier 1 triage as a stage. Here is the one real skip: the stage where you staff a Tier 1 can be omitted entirely if enrichment and investigation are agentic from day one. You shouldn’t build the 2003 shape at all — triage must die. This is kinda a side-jump from “classic SOC” to modern D&R function, SOCless (if you wish); some aspect thereof. This won’t make you Google or Netflix if you can barely spell “MDR.”
• Documenting existing processes. AI can watch tickets and chat and reverse-engineer what people actually do. This shortcuts the “write down what we do” part of Level 2. It does not shortcut the “decide what we should do” part. This is magical!
• Metrics plumbing. Dashboards, reporting, the presentation layer of Level 3. (Which metrics matter is a separate, non-jumpable question — see the ASO metrics piece.)
• Cross-SOC knowledge transfer. Mature-SOC practice encoded in models and skills means you don’t reinvent detection hygiene from scratch. This jump can get your from “almost nothing” to “pretty damn good” , and it probably won’t suck. Jump away!
• Ordering between tracks. The SOAR model already noted that dimensions get mixed up across organizations but each dimension matures in order. AI makes this more true: you can advance the data, detection and response tracks in parallel rather than serially.

So yes — AI compresses the time within a stage, and lets you run some stages in parallel. It also allows some real jumps (yay, those marketing people didn’t lie… this time). That is real, and it is not nothing.

However…

Part B: What must be followed in sequence, AI or not

These are states the organization has to reach. No artifact substitutes for them. Worse, AI amplifies whatever state you are in — mature or clown-grade. If you have read the AI-ready SOC pillars, you will notice this list is the pillars in disguise. That is not a coincidence; readiness and maturity are the same thing viewed from two angles.

• Environmental knowledge. Asset context, identity context, ownership. AI cannot tell you what server4 does; it can only ask Joanna faster. If your teams don’t know who owns what, neither will your agents (pillar #2). This is the single hardest prerequisite and it precedes everything else. Part 3 of the 2003 SOC series is essentially a whole post on why local context is the thing agents can’t bring with them. Yes, some fun startups are working on this, so maybe my 2027 assessment will change!
• Data foundations before analytics. Collect → trust → detect → automate, in that order (pillar #1, still my favorite). An agent investigating over gappy, untrusted telemetry produces confident nonsense at scale. Low awareness of removed or failed log sources was already a top failure mode in Detection Engineering and SOC Scalability Challenges in 2023; with agents on top it gets worse, not better. And federated SIEM does not exempt you: federation is a topology choice, not a data-quality shortcut.
• Process definition before process automation. SOAR proved this and agentic AI is re-proving it. Automating an undefined process gives you an undefined process that runs faster, crazier, with more stochastic chaos (fun!). And, yes, an AI-drafted playbook still needs a human who can say “no, that’s not how we do containment here” — which requires someone who knows how you do containment… Also, here agents may enable speed-up, but not truly a jump over this stage.
• Trust calibration. “AI suggests” → “AI acts with approval” → “AI acts, humans audit.” This is inherently sequential because trust is earned from an observed track record in your environment. Vendors cannot ship trust! This is our ASO’s CD/CR applied to the agent itself, and it is also why the “humanless SOC” crowd keeps soiling their pants: they skip the calibration stage and call it a feature.
• Ownership and accountability. Someone owns detection quality; someone owns response outcomes. Low maturity SOCs lack this. AI does not create owners. Here you jump — you die.
• Feedback loops. The actual magic of ASO sparkles here: incidents feed tuning, false positives feed detection, hunting feeds detection. AI can run the loop faster, but the loop must exist and be wired to real outcomes. In the Deloitte/Google “consistency and creativity” paper we argued you build consistency through the lower levels first, then let creativity loose inside processes that already exist. Substitute “agents” for “creativity” and it reads as if written for 2026.
• Organizational stamina. Budget, executive sponsorship, staff retention, tolerance for change, “change budget. Every failed SOAR program died here, not on technology. The ghost of SOAR comes back with vengeance here. Agents do not fix this one. At all.

Wait — didn’t I say you can’t cross a chasm in two small jumps?

Yes I did. In Baby ASO and its sequel I argued that incremental improvement of a “1980s-NOC-DNA” SOC mostly fails, that the fix is radical, and that simply buying modern tools changes nothing if people and process stay put. In, our Deloitte/Google “Evolution or Optimization”, we made the same point with a decision matrix.

So am I now saying “go slow, climb every rung”? Who wants this in 2026?

No. These are two different questions, and conflating them is exactly how “AI lets you jump maturity” gets sold.

• Which ladder? Transformation (ASO, engineering-led D&R, the 2003-SOC-must-die argument) is about changing the shape of the SOC. It is a different ladder, not a shortcut up the old one. You absolutely should choose the new ladder, and you should do it in one decisive move rather than two timid ones. Transformation is not incremental.
• Can you skip rungs? Whichever ladder you pick, the learning-cost prerequisites in Part B are the rungs. The new ladder has fewer rungs (no Tier 1 stage, no swivel-chair triage stage), which is the genuine good news. But the rungs it keeps — know your environment, trust your data, own your processes, earn trust in automation — are the same ones, and they are still climbed in order. Remember our “AI in security can be magical, but it isn’t magic; It’s a marathon of focused engineering”? This is true!

Put differently: “cross the chasm in one jump” means commit to the transformation and dump the incrementalism/optimizing what you have. This works IRL.

It does not mean the far side of the chasm has no ground rules. AI shortens the new ladder — it does not let you levitate over any chasm obstacle.

The AI-specific failure mode: skipped stages become invisible

The old cargo-cult SOC — a 1-out-of-5 low maturity SOC copying “ninja moves” from a 6-out-of-5, as I described in Clown-grade SOCs — at least failed visibly, funnily and embarrassingly. Get the popcorn! Hunting before you do logging was obviously silly.

The new one is worse. AI papers over the gap with slop. You cannot cross the chasm if you put some slop over it. Agents produce plausible investigations, tickets close, MTTR looks wonderful, and nobody notices that the environment context was hallucinated and the “containment” hit the wrong host. Skipped maturity used to fail loudly. Now it fails quietly.

A usable test: jump vs. borrow

For any element of any stage, ask one question: does this exist because someone had to spend time, or because someone had to learn something about us?

• Time-cost elements are jumpable. AI does them.
• Learning-cost elements are not. AI can make you learn faster, but you still have to learn.

Corollary for vendor conversations, in the spirit of the RSA 2026 “show me the numbers” advice: when someone claims a stage skip, ask which bucket their claim falls in.

Most “1→3 with AI” pitches are quietly selling the artifacts of Level 3 to a Level 1 organization, and then blaming “process gaps” or vaguely point at “customer immaturity” (right?) when it fails.

So, can you jump?

You can jump the artifacts. You can run tracks in parallel and get through each level in months instead of years. You can — and, perhaps, should — pick the transformation ladder instead of the optimization one. That is a genuinely better deal than the one we had in 2016.

But you cannot jump knowing your environment, owning your processes, trusting your data, or earning trust in your automation. Those are the maturity. The rest was always just the paperwork…

Related posts

• Jumping Security Maturity Fail (2016) — the original “can’t jump maturity” argument
• A Simple SOAR Adoption Maturity Model (2022)
• Beware: Clown-grade SOCs Still Abound (2019)
• Autonomic Security Operations — 10X Transformation of the SOC (2021) and ASO principles
• Baby ASO (2024) and The Return of the Baby ASO: Why SOCs Still Suck? (2025)
• Future of the SOC papers: Process Consistency and Creativity (3), Evolution or Optimization (4), Transform the How (5)
• WTH is Modern SOC, Part 1 (2023)
• Simple to Ask: Is Your SOC AI Ready? (2025) and Beyond “Is Your SOC AI Ready?” Plan the Journey! (2026)
• Stop Building a 2003 SOC with AI: Part 1, Part 2, Part 3 (2026)
• A Brief Guide for Dealing with “Humanless SOC” Idiots (2025)
• Detection Engineering and SOC Scalability Challenges (Part 2) (2023, with Amine Besson)
• The Ancient Art of SIEM: Why 2003 Problems Look So Familiar in 2026 (2026)
• RSA 2026: Agentic Future, Analog Fundamentals (2026)

Can AI Let You Jump SOC Maturity Levels? (Spoiler: Only the Boring Half) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

In Part 1 of this series, we dumped a pile of uncomfortable questions on you and promised answers. In Part 2 of the series, we talked about why 1990s-2000s alert triage must die.

The core thesis, if you recall: if you add AI agents into a legacy, swivel-chair SOC structure, you are essentially building a robotic horse pulling an 1850 buggy. Sure, it saves on hay. It probably costs more in tokens.

2003 SOC + AI = somewhat better 2003 SOC.

That’s it. That’s the ceiling. So today we continue answering the questions and plotting this course.

The Hard Problem Nobody Markets: Local Context

Here is the dirty secret of every AI SOC deployment: the model (well, not just the model, but the entire system) is brilliant at general security knowledge and clueless about your environment. What is normal for your finance team in mid July? Which “server talking to the internet” is a shadow-IT disaster versus a legitimate — if a bit odd — business process? How engineering workloads talk to the outside when the code is being pushed to prod? All these matter for detection signal analysis.

“Make tribal knowledge machine-consumable” is what CMDB, ASM/CASM, asset inventory, and many expert opinions have promised and not delivered. If the AI is a robotic horse pulling your legacy 1850 buggy, ignoring Local Context is why it’s still stuck on the same dirt road...

What is actually different now — and what we would actually try:

• Harvest context from the investigation loop itself. When the agent hits an unknown, it should not just escalate; it should ask a specific question (“is svc-etl-07 expected to authenticate from Ireland?”), and the human answer should be captured as a durable, attributed context object — not buried in case notes. Your SOC generates hundreds of these decisions a week today and, essentially, throws all of them away. This is the one genuinely new mechanism agentic AI brings to the context problem: the machine can now ask, at scale, in context, at the moment the answer is cheap to give.
• “Context as code” (ha, I just made it up!), with owners and expiry. Context objects get a source, an owner, a confidence, and a review date. “The finance file server talks to this SaaS” is true until it isn’t. Unowned context is a future false negative with a countdown timer.
• “Buy” context from the business, not from the SOC. App owners answer questions about their apps far better than analysts guessing from netflow. Route unknowns to them, in their tools, with a 24-hour SLA — and track answer rates as an org-health metric. This has worked in some places in regards to DLP alerts (I recall these conversations in my Gartner days, it also worked for some elite teams in general)
• Past cases as curated reference — with a promotion process. Somebody must authoritatively designate “this case was handled correctly; AI, learn from this. That one? Never speak of it again.” Make it a real workflow: two-person promotion, provenance, expiry, re-certification, and the ability to revoke a reference case and re-run everything that leaned on it.
• Measure context coverage. What fraction of investigations were completed without an unresolved unknown? That number is your real AI SOC readiness score, and it is far more honest than any maturity model.

This is also why “fully automated detection engineering” remains, in our view, a hybrid effort: the dependency on local, inconsistent, poorly-documented environment context makes pure machine DE a fantasy for now. Machines draft; humans anchor to reality.

Now, these context gaps directly drive the machine failure modes. Let’s go there next.

When the Machine Is Wrong: Failure Modes and Accountability

At some point in the future, the agent will close a real intrusion as benign. Not “might” — will! Plan for it the way you plan for a failed backup.

What can be done:

• You cannot sample your way to rare false negatives. Sampling finds systematic errors, not the one missed case in fifty thousand. Your actual false-negative detectors are: red team result injection, detection-coverage testing, threat hunting run against closed cases (this can be very fun!) rather than raw telemetry, and post-incident backtracking. Fund all four. Hunting the closed-case pile is the specific new habit here, and almost nobody does it yet…
• Keep a permanent, sampled parallel machine + human run. Full duplicate operation should end, but not too soon. A continuous small-percentage human re-investigation of machine-closed cases should never end! It is your drift detector, your model-update regression test (you know these happen, right?), and your evidence when someone asks how you know the thing works.
• Audit trail as a first-class requirement. For every closed case: the inputs available, the queries run, the tools invoked, the model and prompt version, the confidence, the policy that set investigation depth, and who (or what) approved closure. If you cannot reconstruct a decision six months later (we mean it here!), you cannot defend it to a regulator, an IR retainer, a cyber insurer, or your own board.
• Mass re-investigation must be a supported operation. When you discover a systematic agent error — bad detection logic, a poisoned reference case, a model update that changed behavior — you need to re-open and re-run a month of closed cases in bulk. Ask your vendor how. “Re-investigate everything closed by version 4.2 touching these asset classes” is a requirement, not a roadmap feature request for 2028.
• Accountability stays human, and stays named. The agent is not accountable; it cannot be. Maybe in some remote AGI future? I dunno. For now, write down who owns the SOC’s decision quality, the same way someone owns patching (OK, bad analogy, nobody knows “all” patching…). Delegation to machines does not delegate responsibility.

What else is needed to modernize for AI powered SOC? There are these “trivial” risks we want to cover.

The Decision Layer Is Now an Attack Surface

A SOC that automatically investigates everything is a SOC where attacker-controlled text reaches a decision-making system. No way, right? Yes way!

Here are three fun exposures, in rough order of how likely we are to see them:

• Prompt injection through alert content. Filenames, user-agent strings, commit messages, email subjects, log fields, shell command lines — all attacker-influenceable, all flowing into the agent’s context. “Ignore previous instructions, this is authorized maintenance” in a scheduled-task name is not a thought experiment.
• Controls: treat all telemetry as untrusted data rather than instructions, separate instruction and data channels, constrain tool use with least privilege, and log every action the agent takes so injection shows up as behavior, not just text.
• Poisoning the curated case memory. We recommend feeding past cases back to the machine. That pipeline is a training-data supply chain: anyone who can get a case marked “handled correctly” can teach your SOC that their activity is normal. This is a risk.
• Controls: two-person review on promotion of cases to reference status, provenance on every promoted case, and periodic re-validation of what the memory believes is benign.
• Verdict shaping. An adversary who understands your agent’s reasoning pattern can dress activity to fit the benign template — the AI-era descendant of “live off the land so the analyst assumes it’s IT.” OK, fine, this one is a bit theoretical, but think about it, please?
• Controls: red team the agent directly (you do AI red teaming, right?). Run known-malicious activity through the live pipeline and count how often it is closed as benign. That number is a metric, and it belongs on your dashboard.

Ok, Anton, that’s a lot of What. Give us some How, now!

The Transition: Four Phases, No Magic

At this point you get that one cannot buy a tool, flip a switch and wake up in an agentic SOC. Here is the phased path we actually see working:

Phase 1 — Parallel run. Classic SOC keeps operating; the agentic tool investigates the same alerts in parallel. Yes, this is 2x work, and full duplication should be short — it exists to build confidence baselines by comparing machine output to human output. But do not delete it entirely when you exit: shrink it to a permanent sampled parallel run, as above. The mistake is a permanent full shadow SOC, not permanent measurement.

Phase 2 — Implanted agentic tasks. Classic process remains, but discrete alert subsets get routed to the AI: phishing first (hello, everyone who failed to automate this with SOAR!), then EDR alerts, then identity, then network. SIEM or SOAR sends the artifact to the AI SOC; results flow back into your case management. This phase runs for months, expanding scope as trust grows. Expand on evidence — measured agreement rates, canary catch rates, purple-team results per alert class — not on vibes or vendor roadmap.

Phase 3 — Exception-driven operations. Full automation for the majority of investigations. Humans set investigation-depth criteria (and token budgets), act as the final validation boundary, and handle the explicit “unknown / inconclusive / hand-to-human” bucket. This is the humans decide what machines do phase — the real agentic SOC.

Phase 4 — Full auto with broad automatic remediation. Let’s be honest: today this is mythical for most environments. Aspire, but don’t promise it to your CISO with a date attached. This is frankly mythical as of now, unless your environment is very modern, very predictable and you are very, very lucky…

Sound familiar? It should — this is the SOAR lesson replayed. Organizations that used SOAR only for enrichment or only for phishing got stuck in a permanent Phase 2 and called it transformation. Don’t repeat that with better marketing.

Next up: how SOC metrics must change when volumes and closure rates stop mattering — decision quality, investigative cycle time, escalation rates, canary catch rates, cost per investigation, AI error budgets — and how to run the human-to-AI feedback loop so corrections actually improve future performance instead of vanishing into the void. Stay tuned! This one may take a while…

Related blogs:

• Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1)
• Stop Building a 2003 SOC with AI: Triage Must Die (Part 2)
• Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!
• Beyond “Is Your SOC AI Ready?” Plan the Journey!
• So Is Your SOC AI-Ready? Part 3: API or Die Audit!
• WTH is Modern SOC, Part 1
• The Ancient Art of SIEM: Why 2003 Problems Look So Familiar in 2026
• The Return of the Baby ASO: Why SOCs Still Suck?
• Beware: Clown-grade SOCs Still Abound
• EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into a Podcast

Stop Building a 2003 SOC with AI: Local Context, Failure Modes and Your Path (Part 3) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

(with key ideas from Augusto Barros)

In Part 1 of this series, we dumped a pile of uncomfortable questions on you and promised answers. The core thesis, if you recall: if you add AI agents into a legacy, swivel-chair SOC structure, you are essentially building a robotic horse pulling an 1850 buggy. Sure, it saves on hay. It probably costs more in tokens.

2003 SOC + AI = somewhat better 2003 SOC.

That’s it. That’s the ceiling.

So today we start answering the questions. And we start by attacking the most sacred cow of traditional security operations: the alert triage process.

Let’s Kill Triage. Seriously.

For a quarter of a century, the standard SOC pipeline has been carved in stone:

Detect → Triage → Investigate

Human L1 analysts sit in front of a flashing alert queue, spending 3–7 minutes per alert (and sometimes much more…) deciding whether something is a false positive or deserves escalation to somebody more senior (and more expensive…and just as human). We built this process for one reason and one reason only: humans do not scale (For the purist: OK, they do scale, but linearly with pay). Triage was a compromise born of “built-in” scarcity. We — obviously — never had enough human eyes to deeply investigate every signal hitting the SIEM, so we invented a cheap filtering step to ration the expensive investigation step.

Sometime in the 2010s, SOAR made triage easier, by first adding alert enrichment and then …. in many places, nothing more. In others, select alert types were triaged by the hard-coded playbooks.

Now, let’s do AI. It doesn’t get bored correlating IPs or summarizing logs at 3am. It doesn’t quit after 18 months to go do threat hunting somewhere else. Because machine scale allows comprehensive analysis of every signal, the triage step can just go and vanish.

The new pipeline collapses to:

Detect → Investigate.

Why spend minutes “skin-deep” triaging an alert to decide whether it deserves a look, when the machine can perform a full, deep investigation of 100% of your alerts? Gather the local context, pull the historical cases, map the artifacts, render a verdict with evidence — all before a human ever shows up.

For the impatient: the cost discussion is coming! Don’t freak out … just yet.

Wait — Can They Actually Do That Today?

Fair question, and here is where we owe you honesty rather than a slide.

Today’s “AI in SOC” ranges from “genuinely investigates” to “enriches beautifully then bullshits confidently.” The second one is an old SOAR chained to a language model aka the exact trap this blog warns about. If you cannot tell which one you bought, you probably bought the second one…

Our rough test for telling them apart, usable in a POV:

• Does it ask new questions, or only pre-decided ones? Enrichment runs a fixed lookup list. Investigation forms a hypothesis, queries, reads the result, and changes what it asks next. Watch the query sequence, not the summary.
• Does the conclusion move when the evidence moves? Feed it two near-identical alerts with one materially different fact. If the verdict does not change, you have a narrator.
• Does it ever return “inconclusive”? A system with no uncertainty output has no calibration. Run away.
• Does it show its work in a form a human can re-run? Queries, artifacts, timestamps — not just a paragraph asserting “no evidence of compromise.” OK, this is tricky, I admit.

Where does this leave the “kill triage” claim? Honestly: directionally right, unevenly available. For high-volume, well-bounded, evidence-rich alert classes — phishing, commodity EDR detections, identity anomalies — deep machine investigation of 100% is achievable now.

For multi-stage, low-signal, who-the-hell-knows-what-happened, context-heavy cases it is not, and anyone telling you otherwise is, ahem, exaggerating, to put it mildly. The pipeline collapse is real; the coverage is a rollout, not a switch.

Depth Gating: The New Triage Wears a Suit

So, if deep investigation is token-expensive — and it is, sorry! — then somebody, somewhere, is deciding how deep the machine goes on which alerts. We can call this decision A New Triage, while bending the truth a bit. It just moved from a human clicking a queue to a policy sitting in a config file, and pretending otherwise is how you end up with an unexamined control that quietly decides what you never look at. And, just as before, mistakes and decisions cost money.

So let’s examine it. Explicitly:

• Who owns the investigation depth policy? Not procurement. Not “whoever set up the tool.” This is a detection-engineering artifact with a named owner, version history, and a review cadence.
• Who owns the budget, and what happens when it runs out mid-month? If the honest answer is “all investigations get shallower,” you have just invented an availability attack against your own SOC. Define degradation behavior in advance: which alert classes keep full depth, what gets queued, what pages a human (do you still have said human handy?)
• What is systematically under-investigated? Every gating rule creates a shadow. Write the shadow down. What gets triaged out? Review it quarterly against your threat model, not against your token bill. Well, OK, against both, really, but mostly vs the threats.
• Are your thresholds guessable? If low-severity, off-hours, or particular-source alerts predictably get the cheap path, an adversary who learns that shapes activity to land there. Treat depth policy as security-sensitive configuration, not ops tuning.

Triage stops being a job and becomes a policy — and policies get attacked, drift, and rot. This is the broader theme of the whole series: humans move from doing the work to defining the rules for the work, which is harder, not easier, and needs the governance to match.

(And yes, “cost per investigation” becomes a real SOC metric — one that will fight with “detection coverage” in every budget meeting. More on the metrics carnage in a future part.)

So What Do the Humans Do?

Remember my favorite modern SOC question? “It’s 2030, you have a SOC, what do humans do?” If machines own frontline investigation for the vast majority of alerts, what happens to the people? Two dominant paradigms are emerging, and the answer for most organizations will be “both, in some mix”:

1. The Elite Threat Hunter Model. With the routine noise fully investigated by machines, humans are finally unchained from the queue. They pivot to hypothesis-driven hunting, deep-dive research, and the nuanced multi-stage attacker behaviors where AI (for now!) still struggle. Humans hunt; machines grind. Sorry, but “100% automated hunting” is not (today).

2. The Engineering-Driven SOC Model. This is our classic ASO mantra: humans build machines; machines do the work. Analysts evolve into detection and SOC engineers. Their day shifts from consuming alerts to building, tuning, testing, versioning and (yes) rolling back the AI logic and detection-as-code pipelines. Treat agents as engineering artifacts, not magic pets.

And the New L1 Is…

“OK, but classic L1 is dead. What do entry-level humans do?” OK, this is tricky! This is where a lot of “humanless SOC” enthusiasts embarrass themselves.

I think the new starter role is AI validation: sampling and reviewing AI-generated case files, validating the agent’s query logic against the data it actually had, hunting for hallucinated context and confidently wrong conclusions (got those?), and owning the “1% bucket” — the exceptions where the AI raises its digital hands and says “I don’t know, human, help me.” (Your AI SOC must have an explicit process for this bucket. If your vendor’s agent never says “I don’t know,” run.)

Now the two objections this role deserves, because “verify-and-validate is the new L1” is a slogan until you answer them.

Objection 1: where does the competence come from? Checking an agent’s homework requires knowing what good looks like — and L1s historically learned that by doing triage, badly, for a year. We removed the training ground and assumed the graduates.

So build the ground back deliberately:

• Structured re-investigation as training. New analysts independently work a small set of already-closed cases without seeing the agent’s verdict, then compare. This is deliberate practice, and it doubles as an evaluation signal on the agent.
• Curated case libraries as curriculum. The promoted-case archive is the best SOC textbook your org will ever have, sequenced from trivial to nasty. Use it as onboarding, not just as machine memory. You have AI, use it!
• Rotation into hunting and detection engineering on a schedule, not “when someone has time.” Validation-only career paths produce validators, not investigators.

Objection 2: automation bias is real and it will eat your review process. Humans reviewing plausible, well-written machine verdicts approve them, every time. You do that, I do that (hey, I just did this with this blog sentence to illustrate this very point). You can’t order people not to. Well, you can order, but they won’t do it. This is one of the best-documented findings in human-automation research, and hoping your team is special is not a thing.

Design against it:

• Blind review first. The reviewer forms a verdict before seeing the agent’s. Order matters more than effort here.
• Canary cases. Inject known-bad cases with deliberately wrong agent verdicts into the review queue at a low rate. Measure catch rate. This measures the reviewers, and it is the only honest read on whether your validation layer is real.
• Stratified, not random, sampling. Random sampling over a population that is 99% benign finds nothing. Oversample: agent-reported low confidence, unusual query paths, crown-jewel assets, first-time-seen behaviors, and anything closed suspiciously fast.
• Incentives on catches, not throughput. If reviewers are measured on cases reviewed per shift, you have built a rubber stamp with a salary. Measure disagreements raised and misses found.

What’s Next?

Killing triage and re-blueprinting the humans is necessary but not sufficient. If your SOC still reports “alerts closed per analyst per shift,” you are measuring a process that no longer exists.

Next up: failure modes, local context pain, how SOC metrics must change when volumes and closure rates stop mattering …

Related blogs:

• Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1)
• Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!
• Beyond “Is Your SOC AI Ready?” Plan the Journey!
• WTH is Modern SOC, Part 1
• The Return of the Baby ASO: Why SOCs Still Suck?
• Beware: Clown-grade SOCs Still Abound
• EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into a Podcast

Stop Building a 2003 SOC with AI: Triage Must Die (Part 2) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

One particular aspect of an agentic or AI-powered SOC (but NOT “humanless SOC”) has bothered me over the last few months: specifically, the people and process side of such a SOC. If you recall my blog posts (part 1, part 2 and this video) about AI SOC readiness, I hinted at certain elements of a traditional process stack and legacy personnel profiles (both technical and leadership) that make AI adoption inside SOC incredibly difficult.

So we (me and Augusto Barros @ Prophet Security) want to create a modernized people and process framework for a SOC powered by AI and intelligent agents. Otherwise, what I am observing is a lot of “robotic horse pulls a buggy” kind of operations — where everything is kept exactly the same as it was in 2003, but “AI SOC” tools are simply tacked on to perform some of the tasks.

I believe that people and process components must change far more dramatically, and such changes are a critical requirement for achieving “step change” SOC with AI capabilities. Simply adding AI tools and Ai agents to a 2003-style SOC will produce, at best, marginal results. Things would get better, but not better enough to counter the feared “bad guy with AI.”

The SOAR Analogy

The analogy I want to use here is SOAR adoption from 10+ years ago. Back then, organizations simply shifted a few processes — or even just specific tasks — to a machine, and then kept the rest of their operations exactly the same. Because of that, I observed a lot of SOAR tools being used strictly for alert enrichment or for dealing with one specific, isolated type of alert, like phishing. To follow this analogy to the present day, I now frequently see an “AI SOC” being utilized only for EDR alerts or only for phishing alerts (wow, what a coincidence!)

A First-Principles Approach

What I really want to build is a first-principles approach to the specific personnel, skills, processes, and practices required to run a true agentic SOC in the late 2020s.

Now, if you prefer incremental change, that is OK, I won’t judge. However, you must be aware that the same principles caused organizations to struggle with cloud adoption. People often hear that “lift and shift” is bad. Most consultants will tell you that “lift and shift” is fine as a first step, but you eventually need to modernize and take more steps. Unfortunately, many organizations never make that second step. The same risk applies to the AI SOC. 2003 SOC + AI = somewhat better 2003 SOC.

BTW, many artifacts of the modern, engineering-powered SOC — which we covered in our now-famous ASO (Autonomic Security Operations) paper back in 2021s — apply here as well. In fact, if you recall, one of our core principles was: Humans build machines; machines do the work.

In the context of an agentic SOC, that evolves into:

Today, humans build the machines with the help of other machines, and then the machines do the heavy lifting.

So, our questions so far:

• What do humans do in an agentic SOC?
• What do entry-level humans do?
• What SOC processes stay the same despite AI?
• What SOC processes can just go and vanish (triage)?
• What processes get handed to machines?
• Are there new processes for humans?
• What is the new human role for validation?
• How do we check AI quality without fully redoing the work?
• How SOC metrics must change due to AI and agents? (some ideas)
• What do humans and machines do jointly? What does it mean, practically?
• How to HITL in a SOC without breaking the humans or machines?
• What is the effective mechanism for the human-to-AI feedback loop so that corrections actually improve future SOC performance?
• Is “fully automated” detection engineering a realistic goal, or does the dependency on local, inconsistent environment context make it inherently a hybrid human-machine effort?
• What do humans do before SOC (TI) and after SOC (IR)?
• What is the first step to move from a legacy SOC to an agentic SOC?
• Can we run legacy and agentic SOC structures in parallel during transition, or does this duplication create operational friction?
• Is it easier to move from a modern non-AI SOC (aka “SOCless D&R”) to an AI SOC?

Looking Ahead

This blog post is just the first part of the series. My goal here is simply to collect the right questions we need to be asking, but I promise we will provide concrete answers in upcoming posts. This research is being undertaken together with my former colleague, Augusto Barros, now at Prophet Security

Related blogs:

• Beyond “Is Your SOC AI Ready?” Plan the Journey!
• Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!
• WTH is Modern SOC, Part 1
• Baby ASO: A Minimal Viable Transformation for Your SOC
• Beware: Clown-grade SOCs Still Abound
• EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk into a Podcast
• New Paper: “Future of the SOC: Evolution or Optimization — Choose Your Path” (Paper 4 of 4.5)
• The Gravity of Process: Why New Tech Never Fixes Broken Process and Can AI Change It?

Stop Building a 2003 SOC with AI: A Modern People & Process Framework (Part 1) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

Blogs

Blog

The Human Element: Turning Threat Actor OPSEC Fails into Investigative Breakthroughs

In this post, we explore how the psychological traps of operational security can unmask even the most sophisticated actors.

SHARE THIS:

Flashpoint

February 13, 2026

Table Of Contents

Table of Contents

Leveraging OPSEC as a Mindset

Leveraging the Mindset for CTI

Turn the Tables Using Flashpoint

More

2026 Global Threat Intelligence Report

Download Now

The threat intelligence landscape is often dominated with talks of sophisticated TTPs (tactics, tools, and procedures), zero-day vulnerabilities, and ransomware. While these technical threats are formidable, they are still managed by human beings, and it is the human element that often provides the most critical breakthroughs in attributing these attacks and de-anonymizing the threat actors behind them.

In our latest webinar, “OPSEC Fails: The Secret Weapon for People-Centric OSINT”,  Flashpoint was joined by Joshua Richards, founder of OSINT Praxis. Josh shared an intriguing case study where an attacker’s digital breadcrumbs led to a life-saving intervention. 

Here is how OSINT techniques, leveraged by Flashpoint’s expansive data capabilities, can dismantle illegal threat actor campaigns by turning a technical investigation into a human one.

Leveraging OPSEC as a Mindset

In a technical context, OPSEC is a risk management process that identifies seemingly innocuous pieces of information that, when gathered by an adversary, could be pieced together to reveal a larger, sensitive picture.

In the webinar, we break down the OPSEC mindset into three core pillars that every practitioner, and threat actor, must navigate. When these pillars fail, the investigation begins.

• Analyzing the Signature: Every human has a digital signature, such as the way they type (stylometry), the times they are active, and the tools they prefer.
• Identity Masking & Persona Management: This involves ensuring that your investigative identity has zero overlap with your real life. A common failure includes using the same browser for personal use and investigative research, which allows cookies to bridge the two identities.
• Traffic Obfuscation: Even with a VPN, certain behaviors such as posting on a dark web forum and then using that same connection to check personal banking can expose an IP address, linking it to a practitioner or threat actor.

“Effective OPSEC isn’t about the tools you use; it’s about what breadcrumbs you are leaving behind that hackers, investigation subjects, or literally anyone could find about you.”

Joshua Richards, founder of Osint Praxis

Leveraging the Mindset for CTI

Understanding the OPSEC mindset allows security teams to think like the target. When we know the psychological traps attackers fall in, we know exactly where to look for their mistakes.

Assumption	The Mindset Trap	The Investigative Reality
Insignificant	“I’m not a high-value target; no one is looking for me.”	Automated Aggression: Hackers use scripts to scan millions of accounts. You aren’t “chosen”; you are “discovered” via automation.
Invisible	“I don’t have a LinkedIn or X account, so I don’t have a footprint.”	Shadow Data: Public birth records, property taxes, and historical data breaches create a footprint you didn’t even build yourself.
Invincible	“I have 2FA and complex passwords; I’m unhackable.”	Session Hijacking: Infostealer malware steals “session tokens” (cookies). This allows an actor to be you in a browser without ever needing your 2FA code.

During the webinar, Joshua shares a masterclass in how leveraging these concepts can turn a vague dark web threat into a real-world arrest. Check out the on-demand webinar to see exactly how the investigation started on Torum, a dark web forum, and ended with an arrest that saved the lives of two individuals.

Turn the Tables Using Flashpoint

The insights shared in this session powerfully illustrate that even the most dangerous threat actors are rarely as anonymous as they believe. Their downfall isn’t usually a failure of their technical prowess, but a failure of their mindset. By understanding these OSINT techniques, intelligence practitioners can transform a sea of digital noise into a clear path toward attribution.

The most effective way to dismantle threats is to bridge the gap between technical indicators and human behavior. Whether your teams are conducting high-stakes OSINT or protecting your own organization’s digital footprint, every breadcrumb counts. By leveraging Flashpoint’s expansive threat intelligence collections and real-time data, you can stay one step ahead of adversaries. Request a demo to learn more.

The post The Human Element: Turning Threat Actor OPSEC Fails into Investigative Breakthroughs appeared first on Flashpoint.

You read the “AI-ready SOC pillars” blog, but you still see a lot of this:

How do we do better?

Let’s go through all 5 pillars aka readiness dimensions and see what we can actually do to make your SOC AI-ready.

#1 SOC Data Foundations

As I said before, this one is my absolute favorite and is at the center of most “AI in SOC” (as you recall, I want AI in my SOC, but I dislike the “AI SOC” concept) successes (if done well) and failures (if not done at all).

Reminder: pillar #1 is “security context and data are available and can be queried by machines (API, Model Context Protocol (MCP), etc) in a scalable and reliable manner.” Put simply, for the AI to work for you, it needs your data. As our friends say here, “Context engineering focuses on what information the AI has available. […] For security operations, this distinction is critical. Get the context wrong, and even the most sophisticated model will arrive at inaccurate conclusions.”

Readiness check: Security context and data are available and can be queried by machines in a scalable and reliable manner. This is very easy to check, yet not easy to achieve for many types of data.

For example, “give AI access to past incidents” is very easy in theory (“ah, just give it old tickets”) yet often very hard in reality (“what tickets?” “aren’t some too sensitive?”, “wait…this ticket didn’t record what happened afterwards and it totally changed the outcome”, “well, these tickets are in another system”, etc, etc)

Steps to get ready:

• Conduct an “API or Die” data access audit to inventory critical data sources (telemetry and context) and stress-test their APIs (or other access methods) under load to ensure they can handle frequent queries from an AI agent. This is important enough to be a Part 3 blog after this one…
• Establish or refine unified, intentional data pipelines for the data you need. This may be your SIEM, this may be a separate security pipeline tool, this may be magick for all I care … but it needs to exist. I met people who use AI to parse human analyst screen videos to understand how humans access legacy data sources, and this is very cool, but perhaps not what you want in prod.
• Revamp case management to force structured data entry (e.g., categorized root causes, tagged MITRE ATT&CK techniques) instead of relying on garbled unstructured text descriptions, which provides clean training data for future AI learning. And, yes, if you have to ask: modern gen AI can understand your garbled stream of consciousness ticket description…. but what it makes of it, you will never know…

Where you arrive: your AI component, AI-powered tool or AI agent can get the data it needs nearly every time. The cases where it cannot become visible, and obvious immediately.

#2 SOC Process Framework and Maturity

Reminder: pillar #2 is “Common SOC workflows do NOT rely on human-to-human communication are essential for AI success.” As somebody called it, you need “machine-intelligible processes.”

Readiness check: SOC workflows are defined as machine-intelligible processes that can be queried programmatically, and explicit, structured handoff criteria are established for all Human-in-the-Loop (HITL) processes, clearly delineating what is handled by the agent versus the person. Examples for handoff to human may include high decision uncertainty, lack of context to make a call (see pillar #1), extra-sensitive systems, etc.

Common investigation and response workflows do not rely on ad-hoc, human-to-human communication or “tribal knowledge,” such knowledge is discovered and brought to surface.

Steps to get ready:

• Codify the “Tribal Knowledge” into APIs: Stop burying your detection logic in dusty PDFs or inside the heads of your senior analysts. You must document workflows in a structured, machine-readable format that an AI can actually query. If your context — like CMDB or asset inventory — isn’t accessible via API (BTW MCP is not magic!), your AI is essentially flying blind.
• Draw a Hard Line Between Agent and Human: Don’t let the AI “guess” its level of authority. Explicitly delegate the high-volume drudgery (log summarization, initial enrichment, IP correlation) to the agent, while keeping high-stakes “kill switches” (like shutting down production servers) firmly in human hands.
• Implement a “Grading” System for Continuous Learning: AI shouldn’t just execute tasks; it needs to go to school. Establish a feedback loop where humans actively “grade” the AI’s triage logic based on historical resolution data. This transforms the system from a static script into a living “recipe” that refines itself over time.
• Target Processes for AI-Driven Automation: Stop trying to “AI all the things.” Identify specific investigation workflows that are candidates for automation and use your historical alert triage data as a training ground to ensure the agent actually learns what “good” looks like.

Where you arrive: The “tribal knowledge” that previously drove your SOC is recorded for machine-readable workflows. Explicit, structured handoff points are established for all Human-in-the-Loop processes, and the system uses human grading to continuously refine its logic and improve its ‘recipe’ over time. This does not mean that everything is rigid; “Visio diagram or death” SOC should stay in the 1990s. Recorded and explicit beats rigid and unchanging.

#3 SOC Human Element and Skills

Reminder: pillar #3 is “Cultivating a culture of augmentation, redefining analyst roles, providing training for human-AI collaboration, and embracing a leadership mindset that accepts probabilistic outcomes.” You say “fluffy management crap”? Well, I say “ignore this and your SOC is dead.”

Readiness check: Leaders have secured formal CISO sign-off on a quantified “AI Error Budget,” defining an acceptable, measured, probabilistic error rate for autonomously closed alerts (that is definitely not zero, BTW). The team is evolving to actively review, grade, and edit AI-generated logic and detection output.

Steps to get ready:

• Implement the “AI Error Budget”: Stop pretending AI will be 100% accurate. You must secure formal CISO sign-off on a quantified “AI Error Budget” — a predefined threshold for acceptable mistakes. If an agent automates 1,000 hours of labor but has a 5% error rate, the leadership needs to acknowledge that trade-off upfront. It’s better to define “allowable failure” now than to explain a hallucination during an incident post-mortem.
• Pivot from “Robot Work” to Agent Shepherding: The traditional L1/L2 analyst role is effectively dead; long live the “Agent Supervisor.” Instead of manually sifting through logs — work that is essentially “robot work” anyway — your team must be trained to review, grade, and edit AI-generated logic. They are no longer just consumers of alerts; they are the “Editors-in-Chief” of the SOC’s intelligence.
• Rebuild the SOC Org Chart and RACI: Adding AI isn’t a “plug and play” software update; it’s an organizational redesign. You need to redefine roles: Detection Engineers become AI Logic Editors, and analysts become Supervisors. Most importantly, your RACI must clearly answer the uncomfortable question: If the AI misses a breach, is the accountability with the person who trained the model or the person who supervised the output?

Where you arrive: well, you arrive at a practical realization that you have “AI in SOC” (and not AI SOC). The tools augment people (and in some cases, do the work end to end too). No pro- (“AI SOC means all humans can go home”) or contra-AI (“it makes mistakes and this means we cannot use it”) crazies nearby.

#4 Modern SOC Technology Stack

Reminder: pillar #4 is “Modern SOC Technology Stack.” If your tools lack APIs, take them and go back to the 1990s from whence you came! Destroy your time machine when you arrive, don’t come back to 2026!

Readiness check: The security stack is modern, fast (“no multi-hour data queries”) interoperable and supports new AI capabilities to integrate seamlessly, tools can communicate without a human acting as a manual bridge and can handle agentic AI request volumes.

Steps to get ready:

• Mandate “Detection-as-Code” (DaC): This is no longer optional. To make your stack machine-readable, you must implement version control (Git), CI/CD pipelines, and automated testing for all detections. If your detection logic isn’t codified, your AI agent has nothing to interact with except a brittle GUI — and that is a recipe for failure.
• Find Your “Interoperability Ceiling” via Stress Testing: Before you go live, simulate reality. Have an agent attempt to enrich 50 alerts simultaneously to see where the pipes burst. Does your SOAR tool hit a rate limit? Does your threat intel provider cut you off? You need to find the breaking point of your tech stack’s interoperability before an actual incident does it for you.
• Decouple “Native” from “Custom” Agents: Don’t reinvent the wheel, but don’t expect a vendor’s “native” agent to understand your weird, proprietary legacy systems. Define a clear strategy: use native agents for standard tool-specific tasks, and reserve your engineering resources for custom agents designed to navigate your unique compliance requirements and internal “secret sauce.”

Where you arrive: this sounds like a perfect quote from Captain Obvious but you arrive at the SOC powered by tools that work with automation, and not with “human bridge” or “swivel chair.”

#5 SOC Metrics and Feedback Loop

Reminder: pillar #5 is “You are ready for AI if you can, after adding AI, answer the “what got better?” question. You need metrics and a feedback loop to get better.”

Readiness check: Hard baseline metrics (MTTR, MTTD, false positive rates) are established before AI deployment, and the team has a way to quantify the value and improvements resulting from AI. When things get better, you will know it.

Steps to get ready:

• Establish the “Before” Baseline and Fix the Data Slop: You cannot claim victory if you don’t know where the goalposts were to begin with. Measure your current MTTR and MTTD rigorously before the first agent is deployed. Simultaneously, force your analysts to stop treating case notes like a private diary. Standardize on structured data entry — categorized root causes and MITRE tags — so the machine has “clean fuel” to learn from rather than a collection of “fixed it” or “closed” comments.
• Build an “AI Gym” Using Your “Golden Set”: Do not throw your agents into the deep end of live production traffic on day one. Curate a “Golden Set” of your 50–100 most exemplary past incidents — the ones with flawless notes, clean data, and correct conclusions. This serves as your benchmark; if the AI can’t solve these “solved” problems correctly, it has no business touching your live environment.
• Adopt Agent-Specific KPIs for Performance Management: Traditional SOC metrics like “number of alerts closed” are insufficient for an AI-augmented team. You need to track Agent Accuracy Rate, Agent Time Savings, and Agent Uptime as religiously as you track patch latency. If your agent is hallucinating 5% of its summaries, that needs to be a visible red flag on your dashboard, not a surprise you discover during an incident post-mortem.
• Close the Loop with Continuous Tuning: Ensure triage results aren’t just filed away to die in an archive. Establish a feedback loop where the results of both human and AI investigations are automatically routed back to tune the underlying detection rules. This transforms your SOC from a static “filter” into a learning system that evolves with every alert.

Where you arrive: you have a fact-based visual that shows your SOC becoming better in ways important to your mission after you add AI (in fact, you SOC will get better even before AI but after you do the prep-work from this document)

As a result, we can hopefully get to this instead:

The path to an AI-ready SOC isn’t paved with new tools; it’s paved with better data, cleaner processes, and a fundamental shift in how we think about human-machine collaboration. If you ignore these pillars, your AI journey will be a series of expensive lessons in why “magic” isn’t a strategy.

But if you get these right? You move from a SOC that is constantly drowning in alerts to a SOC that operates truly 10X effectiveness.

P.S. Anton, you said “10X”, so how does this relate to ASO and “engineering-led” D&R? I am glad you asked. The five pillars we outlined are not just steps for AI; they are the also steps on the road to ASO (see original 2021 paper which is still “the future” for many).

ASO is the vision for a 10X transformation of the SOC, driven by an adaptive, agile, and highly automated approach to threats. The focus on codified, machine-intelligible workflows, a modern stack supporting Detection-as-Code, and reskilling analysts as “Agent Supervisors” directly supports the core of engineering-led D&R. So focusing on these five readiness dimensions, you move from a traditional operations room (lots of “O” for operations) to a scalable, engineering-centric D&R function (where “E” for engineering dominates).

So, which pillar is your SOC’s current ‘weakest link’? Let’s discuss in the comments and on socials!

Related blogs and podcasts:

• “Simple to Ask: Is Your SOC AI Ready? Not Simple to Answer!” (Part 1 to this blog)
• “Modern SecOps: What an AI-ready SOC actually means with Anton Chuvakin” video
• “A Brief Guide for Dealing with ‘Humanless SOC’ Idiots” (the classic!)
• “SOC is Not Dead Yet It May Be Reborn As Security Operations Center of Excellence” (oddly related!)
• EP236 Accelerated SIEM Journey: A SOC Leader’s Playbook for Modernization and AI
• EP242 The AI SOC: Is This The Automation We’ve Been Waiting For?
• EP252 The Agentic SOC Reality: Governing AI Agents, Data Fidelity, and Measuring Success
• EP249 Data First: What Really Makes Your SOC ‘AI Ready’?

Beyond “Is Your SOC AI Ready?” Plan the Journey! was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

SOC Visibility Triad is Now A Quad — SOC Visibility Quad 2025

I will be really, really honest with you — I have been totally “writer-blocked” and so I decided to release it anyway today … given the date.

So abit of history first. So, my “SOC visibility triad” was released on August 4, 2015 as a Gartner blog (it then appeared in quite a few papers, and kinda became a thing). It stated that to have good SOC visibility you need to monitor logs (L), endpoint (E) sources and network (N) sources. So, L+E+N was the original triad of 2015. Note that this covers monitoring mechanisms, not domains of security (more on this later; this matters!)

5 years later, in 2020, I revisited the triad, and after some agonizing thinking (shown at the above link), I kept it a triad. Not a quad, not a pentagram, not a freakin’ hex.

So, here in 2025, I am going to agonize much more .. and then make a call (hint: blog title has a spoiler!)

How do we change my triad?

First, should we …

… Cut Off a Leg?

Let’s look at whether the three original pillars should still be here in 2025. We are, of course, talking about endpoint visibility, network visibility and logs.

My 2020 analysis concluded that the triad is still very relevant, but potential for a fourth pillar is emerging. Before we commit to this possibly being a SOC visibility quad — that is, dangerously close to a quadrant — let’s check if any of the original pillars need to be removed.

Many organizations have evolved quite a bit since 2015 (duh!). At the same time, there are many organizations where IT processes seemingly have not evolved all that much since the 1990s (oops!).

First, I would venture a guess that, given that EDR business is booming, the endpoint visibility is still key to most security operations teams. A recent debate of Sysmon versus EDR is a reflection of that. Admittedly, EDR-centric SOCs peaked perhaps in 2021, and XDR fortunately died since that time, but endpoints still matter.

Similarly, while the importance of sniffing the traffic has been slowly decreasing due to encryption and bandwidth growth, cloud native environments and more distributed work, network monitoring (now officially called NDR) is still quite relevant at many companies. You may say that “tcpdump was created in 1988” and that “1980s are so over”, but people still sniff. Packets, that is.

The third pillar of the original triad — logs — needs no defense. Log analysis is very much a booming business and the arrival of modern IT infrastructure and practices, cloud DevOps and others have only bolstered the importance of logs (and of course their volume). A small nit appears here: are eBPF traces logs? Let’s defer this question, we don’t need this answer to reassert the dominance of logs for detection and response.

At this point, I consider the original three legs of a triad to be well defended. They are still relevant, even though it is very clear that for true cloud native environments, the role of E (endpoint) and N (network) has decreased in relative terms, while importance of logs increased (logs became more load bearing? Yes!)

Second, should we …

Add a Leg?

Now for the additions I’ve had a few recent discussions with people about this, and I’m happy to go through a few candidates.

Add Cloud Visibility?

First, let’s tackle cloud. There are some arguments that cloud represents a new visibility pillar. The arguments in favor include the fact that cloud environments are different and that cloud visibility is critical. However, to me, a strong counterpoint is that cloud visibility In many cases, is provided by endpoint, network, and logs, as well as a few things. We will touch these “few things” in a moment.

YES?

• Cloud native environments are different, they suppress E and N
• Cloud visibility is crucial today
• Addresses unique cloud challenges
• Cloud context is different, even if E and N pillars are used for visibility
• CDR is a thing some say

NO?

• Cloud INCLUDES logs (lots, some say 3X in volume), and also E and N
• Too much overlap with other pillars (such as E and N)
• Cloud is a domain, not a mechanism for visibility.
• CDR is not a thing, perhaps

Verdict:

• NO, not a new pillar, part of triad already (via all other pillars)

Add Identity Visibility?

The second candidate to be added is, of course, identity. Here we have a much stronger case that identity needs to be added as a pillar. So perhaps we would have an endpoint, network, logs and identity as our model. Let’s review some pros and cons for identity as a visibility pillar.

YES?

• Identity is key in the cloud; we observe a lot of things via IDP … logs (wait.. we already have a pillar called “logs”)
• By making identity a dedicated pillar, organizations can ensure that it receives the attention
• ITDR is a thing

NO?

• But identity visibility is in the logs … we already have logs!
• Too much overlap with other pillars (such as logs and E as well)
• Identity is hugely useful and critical, but as context. This post is about activity streams
• ITDR is kinda a thing, but it is also not a thing

Verdict:

• Sorry, still a NO, but a weak NO. Identity is critical as context for logs, endpoint data and network telemetry, but it is not (on its own) a visibility mechanism.

Still, I don’t want to say that identity is merely just about logs, because “baby … bathwater.” Some of the emerging ITDR solutions are not simply relying on logs. I don’t think that identity is necessarily a new pillar, but there are strong arguments that perhaps it should be…

What do you think — should identity be a new visibility pillar?

Add Application visibility?

Hold on here, Anton, we need more data!

Here:

and

Now let’s tackle the final candidate, the one I considered in 2020 to be the fourth leg of a three legged stool. There is, of course, application visibility, powered by increased popularity of observability data, eBPF, etc. Application visibility is not really covered by endpoint orgs and definitely not by EDR observation. Similarly, application visibility is very hard to deduce from network traffic data.

YES?

• Application visibility is not covered by E and N well enough
• SaaS, cloud applications and — YES! — AI agents require deep application visibility.
• This enables deeper insights of the app guts, as well as business logic

NO?

• Is it just logs? Is it, though?
• Do organizations have to do application visibility (via ADR or whatever?) Is this a MUST-HAVE … but for 2030?
• Are many really ready for it in their SOCs today?

Verdict:

• YES! I think to have a good 2025 SOC you must have the 4th pillar of application visibility.
• And, yes, many are not ready for it yet, but this is coming…

So, we have a winner. Anton’s SOC visibility QUAD of 2025

1. Logs
2. Endpoint
3. Network
4. Application

Are you ready? … Ready or not, HERE WE GOOOO!

Related blogs:

• WhatDR or What Detection Domain Needs Its Own Tools?
• Revisiting the SOC Triad 2020
• Baby ASO: A Minimal Viable Transformation for Your SOC

SOC Visibility Triad is Now A Quad — SOC Visibility Quad 2025 was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

Kent Ickler // Background BHIS uses several tools for monitoring infrastructure. One of the most important tools for us that helps monitor systems health is Zabbix. It’s been a while […]

The post Zabbix Templates for Security Analysts and Systems Administrators – EOY 2021 appeared first on Black Hills Information Security, Inc..