βI Donβt Need a SIEM, I Have Ollama!β (Yes, You Do!)
Warning: this blog is an experiment. The idea for a blog came from a real conversation I had with somebody. The content is researched, debated, refined, βhardenedβ by a Gemini skill I created (using Gemini 4 Argon). Then Claude Opus 5.5 performed a peer review, and tore some parts to shreds. Then rewrote them. And then I very lightly edited it, and tuned a few jokes. I replaced a food fight joke with a pillow fight joke, specifically. Let me know if this sucks, and I will stop.Β Promise!
β
A security peer recently shared a proposal they had just heard in the wild: instead of paying for a SIEM, why not load several days of raw log data into a locally run open-weights model, ask it βwhat looks risky?β, and review whatever the model highlights?
No SIEM ingest tax! No cloud token bills! No SIEM license fees! No writing correlation rules! No tuning anomaly detection! Just a gaming GPU under a desk and pure, artisanal AI-native vibes. Ahhh,Β magic!

Wait. Really?
My first instinct, forged by 24 years of staring at logs since I first met a SIM in January 2002, was to laugh, pour a drink, and move on. Surely everyone knows this is a terrible idea, right?Β Right?!
Wrong.
If an engineer, an IT director, or a cost-cutting CFO pitches βlocal LLM as a poor manβs SIEMβ in your organization, you cannot just roll your eyes and mutter βhallucinations.β People keep inventing this idea for real reasons. SIEM ingest pricing still hurts: some vendor cost explainer works out 5 TB per day at $2 to $4 per GB as $3.6 to $7.3 million a year on ingestion alone. Detection engineering is still hard work. And yes the AI hype machine has spent two years promising executives that models can βreason over any unstructured data.β
So letβs dissect why dumping a week of raw logs into a local model breaks four basic laws of βoperational physicsβ, and then carve out the narrow 10% where a local model does help with logs. The short version fits on a sticker: the LLM writes the parser, it never runs as the parser. And the twist is that the problem here is βraw,β notΒ βlocal.β
Token Arithmetic and Naive Chunking (Or: How Big Do You Think 7 Days of LogsΒ Is?!)
Letβs start with grade-school arithmetic, because clearly nobody pitching this architecture has ever opened a calculatorβ¦
Forget the enterprise. Take a tiny environment, a single server rack generating a measly 1 GB of raw text logs per day. Seven days of logs is 7 GB of text, or roughly 1.75 billion tokens at 4 characters per token. That is generous, since IP addresses, GUIDs and hex strings tokenize worse than English (Wow, really? Yes, reallyβββat least Claude thinks so! He would know,Β right?)
Now look at that βfreeβ 8B or 70B parameter model on your GPU workstation. Its usable context window, before the KV cache devours your VRAM and the fans sound like a Boeing 747 on takeoff, is maybe 32K to 128K tokens. You are shoving 1.75 billion tokens through a 128K keyhole, which is about 13,700 full context windows. Painful enoughΒ yet?
Well, it gets worse. Assume your box reads a generous 2,000 tokens per second (benchmark your own, Iβll wait). Reading those 1.75 billion tokens takes about 10 days. You need 10 days to read 7 days of logs, so you never catch up! Scale that to the 5 TB per day example above and it is about 20 years of GPU time for each day of logs. Hilarity ensues! Replacing a deterministic stream processor with a token-by-token text predictor is like replacing your municipal water pipes with a guy carrying a silver espresso spoon from a well 10 milesΒ away.
βFine, Anton,β the DIY enthusiast says, βweβll chunk the logs into 2,000-line batches and loop the local model over them all weekend!β Or we will get a bigger boatβ¦ ehβ¦ server rack to run theΒ model.
Chunking naively, by just loading the data in batches, fails hilariously. Here isΒ how:
- Attack chains get cut. The phish on Tuesday lands in one batch, the lateral movement on Thursday lands thousands of batches later, and no prompt ever sees both. Each batch gets judgedΒ alone.
- Merging the summaries does not bring the signal back. By the time you summarize the summaries, the one quiet line was already dropped in round one. So βnaive+β fails asΒ well.
- Position inside the batch matters. Liu et al.βs βLost in the Middleβ found that models do best when the relevant information sits at the start or end of a long context and degrade sharply in the middle. They tested question answering and key-value retrieval, not logs, but picture a quiet privilege escalation on line 1,482 of your batch. Worst seat in theΒ house.
Researchers keep running into this. Cai, Ryan and Karie note that iterative approach is limited by context capacity and that map-reduce style summarization is computationally heavy (and requires some luck). Even so, that is a proof of concept on up to 20,000 rows per dataset, and they say enterprise volumes remain to be validated.
So smarter chunking exists (reminder: it is not magic, in this area nothing truly is): parse, cluster, then retrieve by entity and time. At that point you are building the first half of a SIEM. Congratulations!
As I wrote in The Ancient Art of SIEM, cost pain is eternal and only the currency changes. In 2003 you ran out of hardware, in 2015 you ran out of cloud budget, and in 2026 you run out of tokens andΒ VRAM.
The Input-Centric Time Machine: Back to 2002 BatchΒ Review
Notice the most revealing phrase in this idea: βload 1 to 7 days of log dataβ¦ and then review what the model highlights.β
That is not real-time detection. That is batch-mode log archeology! When intruders can go from initial access to data theft in hours (soon: minutes for some incidents), reviewing a week of logs in a batch means you are running a weekly book club about how you got breached last Wednesday. But heyΒ β¦ is your SIEM faster? Well, it shouldΒ be!
Worse, asking an LLM to βhighlight what looks riskyβ violates the most important lesson of two decades of detection engineering. In Output-driven SIEM, 13 years later, I revisited my favorite 2011 mental model: think about what you want to see at the output before you collect, and AI does not change this. Asking a model to βfind riskβ in raw logs with zero detection engineering is the ultimate input-centric anti-pattern. It failed with ML in mid-2010s, and LLM is not different (in thisΒ regard).
Now, about determinism. The lazy objection is βthe answers change on every run,β and a sharp engineer will reply βso pin the seed.β Fair enough! Run-to-run randomness is an engineering problem: Thinking Machines got 80 unique completions out of 1,000 at temperature 0, then 1,000 identical ones after fixing their kernels. But is it actually bad? Maybe if you run 5 times, you will find 2x more βfunβ things. But heyΒ β¦ also probably at least 5x moreΒ noise.
The harder test is stability. Take the same 48 hours of domain controller logs, shuffle the order or change one irrelevant line, and run it again. Given what we learned, I would bet real money the verdict moves. Well-crafted SIEM rule does not care what order the lines arriveΒ in.
Zero Local Context Equals Hallucinated Risk
Why canβt a local model tell an active nation-state intrusion from Tuesdayβs messy IT administration? Because raw logs do not contain your organizational reality! Shocking, eh?
A raw syslog or CloudTrail line says that svc_deploy_04 logged into prod-db-02 at 3:14 AM from an internal jump host and ran an encoded PowerShell script. Is that a five-alarm breach or the legacy billing batch job that Bob in accounting wrote in 2017 and nobody dares to touch? This is a dated example, on purpose. Insert your own fun example involving a shadow AIΒ agent!
In Part 3 of the Stop Building a 2003 SOC series, I argued that the #1 reason AI fails in the SOC is the absence of Context as Code: machine-readable, owned, and expiring knowledge of your assets, business processes, and weird administrative exceptions. In Part 2, we warned about automation bias: humans reviewing plausible, well-written machine verdicts approveΒ them.
Also, the test from Can AI Let You Jump SOC Maturity Levels? applies here: does this exist because someone had to spend time, or because someone had to learn something about us? Writing a parser is a time cost. Knowing about Bobβs job is a learning cost, and no model downloaded from the internet has paidΒ it.
Feed raw logs into a naked local LLM with no CMDB enrichment, no threat intel, no identity graph, and no 90-day baseline, and its only source of βcontextβ is its pre-trained weights. So it flags every scary-looking word (ERROR, DENIED, admin, root, failed) and writes you a convincing fictional thriller about your ownΒ network.
The Confused Deputy Trap: When the Attacker Prompts Your βSIEMβ via User-Agent
Now put on an attacker hat for 30 seconds. What is a logΒ file?
A log file is untrusted, attacker-influenced input. Every public-facing web server, SSH daemon, API gateway, and email relay faithfully records whatever string an outsider sends in an HTTP User-Agent, a failed username field, or a JSONΒ payload.
Log processors have been attacked through logged strings before (hi, Log4Shell!), but that was a bug you could patch. In a raw-log-to-LLM setup there is no parser boundary to patch, because instructions and data share one channel. You just handed the attacker write access to your detection engineβsΒ prompt!
Summaries are also scary, BTW. Pandey and Bhujang found that the classic βignore all instructionsβ override failed completely against the one model they tested, but context manipulation reached 96% success on summarization with no defenses and 38% even with constrained output.
In How to Solve the Mystery of AI Defense in Depth (and my October 2026 collaboration with Usman Chaudhary, The Model Has Preferences, but the Harness Got Controls), I emphasized that a modelβs statistical preferences are never a security boundary, and only a deterministic harness is. Feed unparsed logs straight into a local LLM with no harness, and any kid with curl can blind your review or flood your team with fabricated criticalΒ alertsβ¦
The Problem Is βRaw,β NotΒ βLocalβ
Letβs be fair to the local crowd, because they have aΒ point.
- Local solves a real problem. Logs are full of secrets, PII and internal hostnames. Not shipping them to a third-party API is a legitimate reason to want a model on your own hardware. It alsoβββtheoreticallyβββeliminates an endless stream of unpredictable token costs flowing to a frontier model provider.
- Small models are not the villain. In the same paper, a structured design called Aida (separate agents for investigating, challenging and judging, working through a SIEM) used open-weight Qwen3.5 9B and it was workable for the task theyΒ had.
- βWhat about an agent querying a log store?β Now we are talking! The model forms a question, a database searches 7 days of logs deterministically, and the model reads a small result. That is a very different architecture, and it needs parsed data, a query engine, detections to start from, retention and access control. Sound familiar?
So keep your local model if you want it. What is missing from this proposal is theΒ SIEMβ¦
The Raw-Logs-to-Local-LLM DelusionΒ Matrix
For explaining this to an enthusiastic colleague (or a budget-slashing executive) without starting a pillowΒ fight:
Wait, Is a Local LLM Good for Anything in Log Land? (The 10% Where ItΒ Shines!)
I am a constructive cynic, not a Luddite (At least Gemini 4 Argon thinks so!). As I noted in the SOC maturity post, AI can jump the artifacts, and parsers are on that list (with caveats for tricky log sources). So point your local model at logs as an engineering sidecar, not an ingestion engine:
- Writing the parser, not being the parser. Instead of feeding 50 million lines from a weird legacy mainframe into an LLM, feed 20 sample lines and ask for a regex, OCSF mapping, or pipeline rule. The LLM writes the code once, and deterministic CPU code parses the 50 million lines at wire speed. This mostly works, IΒ swear.
- Post-trigger micro-slice investigation. Once a rule or EDR alert fires on a specific host and 15-minute window, pull 200 pre-filtered, enriched log lines and ask the model for a timeline. Those lines still contain attacker strings, so treat the summary as untrusted and let the model request actions only from a closedΒ list.
- Ad-hoc query translation. Translate an analystβs question into KQL, SPL, or SQL, so the database searches 7 days of logs deterministically. This also mostly works, but someΒ YMMV.
The Lean TeamΒ Reality
What if you are the CISO of a regional insurer or hospital with a small security team, a hybrid stack, strict state regulators, and an MDR provider? When an engineer or cost-conscious VP suggests βletβs dump our noisy firewall logs into a local Ollama box,β they arenβt trying to sabotage your SOC. They have a log source that is too expensive to send to the SIEM, and they want a free way to βkeep an eye on it.β Here is how to solve that without a scienceΒ project:
- Donβt use a GPU box as a poor manβs data lake. Route a noisy source to cheap, searchable object storage, or strip verbose fields in a pipeline before ingest. This is my old two-tier answer (a SIEM plus a broad, cheap log store), and it has agedΒ well.
- Hold your MDR to the same standard. When they brag at the next QBR about βAI-powered log analysis,β ask whether deterministic detections run first or whether it is an LLM wrapper over unparsed telemetry. This point alone is worth the price of reading this blog (A.C.βββthis is me, notΒ AI)
- Channel the DIY energy. Give your smart engineer 50 anonymized lines from your gnarliest legacy application and ask for three high-signal filter rules you can deploy in your real pipeline.
Action Items for SecurityΒ Leaders
Is there anything you need to actuallyΒ do?
- Run a napkin token audit on any AI log proposal (Monday morning). Before the demo starts, ask: βHow many GB per day is the raw source, how fast does the model read, and what deterministic step selects the tiny slice that reaches a context window?β If the answer is βweβll just load it in batches,β kill the project on the spot. Itβs this βjustβΒ word.
- Fix ingest pain with tiering (30-day play). If this idea surfaced because a log source is too expensive for your SIEM, solve that: drop repetitive keepalive chatter, route compliance-only logs to cheap storage, and keep output-driven security data in your detection tier.
- Make βthe LLM writes the parser, never runs as the parserβ a standard (architecture play)βββthat is, if you write parsers at all. Allow LLMs at design time (parsers, mappings, queries from small samples) and at post-alert time (small, bounded, enriched slices), but not as the runtime ingestion or anomaly engine over raw logs. Put it in your MDR requirements tooβ¦ otherwise they will keep whining that it takes 7 weeks to integrate a log sourceΒ forever.
- Test your existing AI log workflows for prompt injection (60-day play). If your SOC or MDR summarizes anything containing attacker-controlled strings, test it in your next purple-team exercise. Do not stop at a naive βignore instructionsβ canary, because passing that proves little. Try persona and context-rewriting payloads, plus payloads split acrossΒ lines.
So, Do You Need aΒ SIEM?
Call it a SIEM or call it a pile of parsers, a query engine, detections and retention. You need those things, and a GPU under a desk does not replace them. By all means run a model next to them, I promise it will beΒ fun!
The part that worries me most is how this fails. A detection rule that misses can be tested, tuned, and re-run. A model that misses hands you a nice summary, and nobody knows what was not in it. As I said in September, skipped maturity used to fail loudly, and now it failsΒ quietly.
We had 23 years of progress, and the best practice is still βhave a plan and donβt ingest garbage.β That goes for your context windowΒ too.
[A.C.βββnote to self: this is v6Β skill]
βI Donβt Need a SIEM, I Have Ollama!β (Yes, You Do!) was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.









