“I Don’t Need a SIEM, I Have Ollama!” (Yes, You Do!)
Warning: this blog is an experiment. The idea for a blog came from a real conversation I had with somebody. The content is researched, debated, refined, “hardened” by a Gemini skill I created (using Gemini 4 Argon). Then Claude Opus 5.5 performed a peer review, and tore some parts to shreds. Then rewrote them. And then I very lightly edited it, and tuned a few jokes. I replaced a food fight joke with a pillow fight joke, specifically. Let me know if this sucks, and I will stop. Promise!
—
A security peer recently shared a proposal they had just heard in the wild: instead of paying for a SIEM, why not load several days of raw log data into a locally run open-weights model, ask it “what looks risky?”, and review whatever the model highlights?
No SIEM ingest tax! No cloud token bills! No SIEM license fees! No writing correlation rules! No tuning anomaly detection! Just a gaming GPU under a desk and pure, artisanal AI-native vibes. Ahhh, magic!

Wait. Really?
My first instinct, forged by 24 years of staring at logs since I first met a SIM in January 2002, was to laugh, pour a drink, and move on. Surely everyone knows this is a terrible idea, right? Right?!
Wrong.
If an engineer, an IT director, or a cost-cutting CFO pitches “local LLM as a poor man’s SIEM” in your organization, you cannot just roll your eyes and mutter “hallucinations.” People keep inventing this idea for real reasons. SIEM ingest pricing still hurts: some vendor cost explainer works out 5 TB per day at $2 to $4 per GB as $3.6 to $7.3 million a year on ingestion alone. Detection engineering is still hard work. And yes the AI hype machine has spent two years promising executives that models can “reason over any unstructured data.”
So let’s dissect why dumping a week of raw logs into a local model breaks four basic laws of “operational physics”, and then carve out the narrow 10% where a local model does help with logs. The short version fits on a sticker: the LLM writes the parser, it never runs as the parser. And the twist is that the problem here is “raw,” not “local.”
Token Arithmetic and Naive Chunking (Or: How Big Do You Think 7 Days of Logs Is?!)
Let’s start with grade-school arithmetic, because clearly nobody pitching this architecture has ever opened a calculator…
Forget the enterprise. Take a tiny environment, a single server rack generating a measly 1 GB of raw text logs per day. Seven days of logs is 7 GB of text, or roughly 1.75 billion tokens at 4 characters per token. That is generous, since IP addresses, GUIDs and hex strings tokenize worse than English (Wow, really? Yes, really — at least Claude thinks so! He would know, right?)
Now look at that “free” 8B or 70B parameter model on your GPU workstation. Its usable context window, before the KV cache devours your VRAM and the fans sound like a Boeing 747 on takeoff, is maybe 32K to 128K tokens. You are shoving 1.75 billion tokens through a 128K keyhole, which is about 13,700 full context windows. Painful enough yet?
Well, it gets worse. Assume your box reads a generous 2,000 tokens per second (benchmark your own, I’ll wait). Reading those 1.75 billion tokens takes about 10 days. You need 10 days to read 7 days of logs, so you never catch up! Scale that to the 5 TB per day example above and it is about 20 years of GPU time for each day of logs. Hilarity ensues! Replacing a deterministic stream processor with a token-by-token text predictor is like replacing your municipal water pipes with a guy carrying a silver espresso spoon from a well 10 miles away.
“Fine, Anton,” the DIY enthusiast says, “we’ll chunk the logs into 2,000-line batches and loop the local model over them all weekend!” Or we will get a bigger boat… eh… server rack to run the model.
Chunking naively, by just loading the data in batches, fails hilariously. Here is how:
- Attack chains get cut. The phish on Tuesday lands in one batch, the lateral movement on Thursday lands thousands of batches later, and no prompt ever sees both. Each batch gets judged alone.
- Merging the summaries does not bring the signal back. By the time you summarize the summaries, the one quiet line was already dropped in round one. So “naive+” fails as well.
- Position inside the batch matters. Liu et al.’s “Lost in the Middle” found that models do best when the relevant information sits at the start or end of a long context and degrade sharply in the middle. They tested question answering and key-value retrieval, not logs, but picture a quiet privilege escalation on line 1,482 of your batch. Worst seat in the house.
Researchers keep running into this. Cai, Ryan and Karie note that iterative approach is limited by context capacity and that map-reduce style summarization is computationally heavy (and requires some luck). Even so, that is a proof of concept on up to 20,000 rows per dataset, and they say enterprise volumes remain to be validated.
So smarter chunking exists (reminder: it is not magic, in this area nothing truly is): parse, cluster, then retrieve by entity and time. At that point you are building the first half of a SIEM. Congratulations!
As I wrote in The Ancient Art of SIEM, cost pain is eternal and only the currency changes. In 2003 you ran out of hardware, in 2015 you ran out of cloud budget, and in 2026 you run out of tokens and VRAM.
The Input-Centric Time Machine: Back to 2002 Batch Review
Notice the most revealing phrase in this idea: “load 1 to 7 days of log data… and then review what the model highlights.”
That is not real-time detection. That is batch-mode log archeology! When intruders can go from initial access to data theft in hours (soon: minutes for some incidents), reviewing a week of logs in a batch means you are running a weekly book club about how you got breached last Wednesday. But hey … is your SIEM faster? Well, it should be!
Worse, asking an LLM to “highlight what looks risky” violates the most important lesson of two decades of detection engineering. In Output-driven SIEM, 13 years later, I revisited my favorite 2011 mental model: think about what you want to see at the output before you collect, and AI does not change this. Asking a model to “find risk” in raw logs with zero detection engineering is the ultimate input-centric anti-pattern. It failed with ML in mid-2010s, and LLM is not different (in this regard).
Now, about determinism. The lazy objection is “the answers change on every run,” and a sharp engineer will reply “so pin the seed.” Fair enough! Run-to-run randomness is an engineering problem: Thinking Machines got 80 unique completions out of 1,000 at temperature 0, then 1,000 identical ones after fixing their kernels. But is it actually bad? Maybe if you run 5 times, you will find 2x more “fun” things. But hey … also probably at least 5x more noise.
The harder test is stability. Take the same 48 hours of domain controller logs, shuffle the order or change one irrelevant line, and run it again. Given what we learned, I would bet real money the verdict moves. Well-crafted SIEM rule does not care what order the lines arrive in.
Zero Local Context Equals Hallucinated Risk
Why can’t a local model tell an active nation-state intrusion from Tuesday’s messy IT administration? Because raw logs do not contain your organizational reality! Shocking, eh?
A raw syslog or CloudTrail line says that svc_deploy_04 logged into prod-db-02 at 3:14 AM from an internal jump host and ran an encoded PowerShell script. Is that a five-alarm breach or the legacy billing batch job that Bob in accounting wrote in 2017 and nobody dares to touch? This is a dated example, on purpose. Insert your own fun example involving a shadow AI agent!
In Part 3 of the Stop Building a 2003 SOC series, I argued that the #1 reason AI fails in the SOC is the absence of Context as Code: machine-readable, owned, and expiring knowledge of your assets, business processes, and weird administrative exceptions. In Part 2, we warned about automation bias: humans reviewing plausible, well-written machine verdicts approve them.
Also, the test from Can AI Let You Jump SOC Maturity Levels? applies here: does this exist because someone had to spend time, or because someone had to learn something about us? Writing a parser is a time cost. Knowing about Bob’s job is a learning cost, and no model downloaded from the internet has paid it.
Feed raw logs into a naked local LLM with no CMDB enrichment, no threat intel, no identity graph, and no 90-day baseline, and its only source of “context” is its pre-trained weights. So it flags every scary-looking word (ERROR, DENIED, admin, root, failed) and writes you a convincing fictional thriller about your own network.
The Confused Deputy Trap: When the Attacker Prompts Your “SIEM” via User-Agent
Now put on an attacker hat for 30 seconds. What is a log file?
A log file is untrusted, attacker-influenced input. Every public-facing web server, SSH daemon, API gateway, and email relay faithfully records whatever string an outsider sends in an HTTP User-Agent, a failed username field, or a JSON payload.
Log processors have been attacked through logged strings before (hi, Log4Shell!), but that was a bug you could patch. In a raw-log-to-LLM setup there is no parser boundary to patch, because instructions and data share one channel. You just handed the attacker write access to your detection engine’s prompt!
Summaries are also scary, BTW. Pandey and Bhujang found that the classic “ignore all instructions” override failed completely against the one model they tested, but context manipulation reached 96% success on summarization with no defenses and 38% even with constrained output.
In How to Solve the Mystery of AI Defense in Depth (and my October 2026 collaboration with Usman Chaudhary, The Model Has Preferences, but the Harness Got Controls), I emphasized that a model’s statistical preferences are never a security boundary, and only a deterministic harness is. Feed unparsed logs straight into a local LLM with no harness, and any kid with curl can blind your review or flood your team with fabricated critical alerts…
The Problem Is “Raw,” Not “Local”
Let’s be fair to the local crowd, because they have a point.
- Local solves a real problem. Logs are full of secrets, PII and internal hostnames. Not shipping them to a third-party API is a legitimate reason to want a model on your own hardware. It also — theoretically — eliminates an endless stream of unpredictable token costs flowing to a frontier model provider.
- Small models are not the villain. In the same paper, a structured design called Aida (separate agents for investigating, challenging and judging, working through a SIEM) used open-weight Qwen3.5 9B and it was workable for the task they had.
- “What about an agent querying a log store?” Now we are talking! The model forms a question, a database searches 7 days of logs deterministically, and the model reads a small result. That is a very different architecture, and it needs parsed data, a query engine, detections to start from, retention and access control. Sound familiar?
So keep your local model if you want it. What is missing from this proposal is the SIEM…
The Raw-Logs-to-Local-LLM Delusion Matrix
For explaining this to an enthusiastic colleague (or a budget-slashing executive) without starting a pillow fight:
Wait, Is a Local LLM Good for Anything in Log Land? (The 10% Where It Shines!)
I am a constructive cynic, not a Luddite (At least Gemini 4 Argon thinks so!). As I noted in the SOC maturity post, AI can jump the artifacts, and parsers are on that list (with caveats for tricky log sources). So point your local model at logs as an engineering sidecar, not an ingestion engine:
- Writing the parser, not being the parser. Instead of feeding 50 million lines from a weird legacy mainframe into an LLM, feed 20 sample lines and ask for a regex, OCSF mapping, or pipeline rule. The LLM writes the code once, and deterministic CPU code parses the 50 million lines at wire speed. This mostly works, I swear.
- Post-trigger micro-slice investigation. Once a rule or EDR alert fires on a specific host and 15-minute window, pull 200 pre-filtered, enriched log lines and ask the model for a timeline. Those lines still contain attacker strings, so treat the summary as untrusted and let the model request actions only from a closed list.
- Ad-hoc query translation. Translate an analyst’s question into KQL, SPL, or SQL, so the database searches 7 days of logs deterministically. This also mostly works, but some YMMV.
The Lean Team Reality
What if you are the CISO of a regional insurer or hospital with a small security team, a hybrid stack, strict state regulators, and an MDR provider? When an engineer or cost-conscious VP suggests “let’s dump our noisy firewall logs into a local Ollama box,” they aren’t trying to sabotage your SOC. They have a log source that is too expensive to send to the SIEM, and they want a free way to “keep an eye on it.” Here is how to solve that without a science project:
- Don’t use a GPU box as a poor man’s data lake. Route a noisy source to cheap, searchable object storage, or strip verbose fields in a pipeline before ingest. This is my old two-tier answer (a SIEM plus a broad, cheap log store), and it has aged well.
- Hold your MDR to the same standard. When they brag at the next QBR about “AI-powered log analysis,” ask whether deterministic detections run first or whether it is an LLM wrapper over unparsed telemetry. This point alone is worth the price of reading this blog (A.C. — this is me, not AI)
- Channel the DIY energy. Give your smart engineer 50 anonymized lines from your gnarliest legacy application and ask for three high-signal filter rules you can deploy in your real pipeline.
Action Items for Security Leaders
Is there anything you need to actually do?
- Run a napkin token audit on any AI log proposal (Monday morning). Before the demo starts, ask: “How many GB per day is the raw source, how fast does the model read, and what deterministic step selects the tiny slice that reaches a context window?” If the answer is “we’ll just load it in batches,” kill the project on the spot. It’s this “just” word.
- Fix ingest pain with tiering (30-day play). If this idea surfaced because a log source is too expensive for your SIEM, solve that: drop repetitive keepalive chatter, route compliance-only logs to cheap storage, and keep output-driven security data in your detection tier.
- Make “the LLM writes the parser, never runs as the parser” a standard (architecture play) — that is, if you write parsers at all. Allow LLMs at design time (parsers, mappings, queries from small samples) and at post-alert time (small, bounded, enriched slices), but not as the runtime ingestion or anomaly engine over raw logs. Put it in your MDR requirements too… otherwise they will keep whining that it takes 7 weeks to integrate a log source forever.
- Test your existing AI log workflows for prompt injection (60-day play). If your SOC or MDR summarizes anything containing attacker-controlled strings, test it in your next purple-team exercise. Do not stop at a naive “ignore instructions” canary, because passing that proves little. Try persona and context-rewriting payloads, plus payloads split across lines.
So, Do You Need a SIEM?
Call it a SIEM or call it a pile of parsers, a query engine, detections and retention. You need those things, and a GPU under a desk does not replace them. By all means run a model next to them, I promise it will be fun!
The part that worries me most is how this fails. A detection rule that misses can be tested, tuned, and re-run. A model that misses hands you a nice summary, and nobody knows what was not in it. As I said in September, skipped maturity used to fail loudly, and now it fails quietly.
We had 23 years of progress, and the best practice is still “have a plan and don’t ingest garbage.” That goes for your context window too.
[A.C. — note to self: this is v6 skill]
“I Don’t Need a SIEM, I Have Ollama!” (Yes, You Do!) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.








