Welcome to Issue #165 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I will be at Hacker Summer Camp next week! I am hosting three events: two panels & a happy hour. I put the details at the end of the post in lieu of the Open Source Section. Come find me and grab a sticker!

• I spent time over the weekend at a lakehouse, and besides the pretty water views, there’s an eerie but beautiful sound I listen for at night. The Loon Wail is such a unique sound, and it’s something my kids get excited about every time they hear it.

• I’ve switched over nearly all of my coding agents to Pi. The customization and the ability to switch between model providers gives me Vim vibes.

detections.ai Enterprise is live. Private, coverage and context aware

A new threat drops and you need to move quickly. Run coverage analysis across your whole detection stack, build detections tuned directly for your environment, and deploy them back. In minutes, not days.

But coverage isn’t a one-time win. Detections drift, IOCs go stale, duplicate rules pile up. Our AI agents catch it while you sleep, so nothing slips through unnoticed, and your team spends its time on real threats, not upkeep.

Book a walkthrough

💎 Detection Engineering Gem 💎

Detections that run but can’t see by Nikhil Satyakrishna

SIEM and detection readiness is a topic I’m hearing more and more about in the research, security product, and internal operations spaces. I seem to link back to my Field Manual post a lot, but the line “There is No Rule without Telemetry” still holds true, but my post fails to address the underlying resilience gotchas. In this post, Satyakrishna addresses these issues and releases an open-source tool to demonstrate how to check for resilience failures.

The four failure modes worth keeping are:

• Disconnected: index pattern matches nothing

• Starved: sources exist but are stale or empty

• Missing fields: the field the rule filters on stopped being populated

• Ingest-lag blind window: events land after the rule’s window has already passed

If you asked me two years ago where the most important failure mode to check for is, it would be “starved”. A log source that doesn’t ship telemetry should be a red flag, but it’s much harder to check for than one would think. Is it not shipping telemetry because there is no telemetry to ship, or is there an underlying issue with the ingest pipeline?

Missing fields are a hard one to crack as well. You’ll need to understand the shape of the logs you are sending in, and make sure that if a log contains a missing field, it’s not due to a broken log source, an upstream provider changing the log envelope, or it’s expected behavior. As I’ve said before, we are becoming closer to SREs than we think :).

I’ve covered xcal’s work for the ingest-lag blind window issue using watermarks, but it’s nice to see another take on this on a log-agnostic use case:

Fraction of events a rule catches as source ingest lag grows, for a rule with a 6-minute lookback running every 5 minutes. The line is the formula, the open points are the simulation, and the shaded area is coverage lost with no error and no alert.

Satyakrishna released an Elastic-specific tool to check for the scenarios listed above plus two more, dubbed deadair. They also linked Elastic’s SIEM Readiness page, so it’s clear to me that research in the detection space is focusing more and more on detection readiness and resilience.

🔬 State of the Art

Agentic incident response notebooks by Kyrre Wahl Kongsgård

I’ve worked on hundreds of incidents at this point in my career, and it almost always falls into the same pattern. A Google Sheet gets shared, and by hour two everyone has 20+ tabs open: SIEM in one, threat intel platform in another, an asset inventory tool in a third, and the infamous scratchpad notes scattered in another. It makes sense, right? Security tools don’t do a great job of collating data, and giving yourself and your team more control over content curation means you need tools everyone can quickly orient to, to visualize the current state of the incident.

Kongsgård & the DNB’s Cyber Defense Center (authors from a previous Gem and building on work I featured almost 60 issues ago) built a super interesting agentic IR workflow on Marimo, where the notebook itself is the case file. Marimo beats Jupyter for this job for reasons that matter specifically to IR, such as Jupyter cells can run out of order, and an output can depend on state from a cell you already deleted, which isn’t helpful when you are trying to maintain chain of evidence or investigative steps.

Marimo gives responders a much more reactive experience. You change the host filter or the time window and every dependent query, join, and chart recomputes from the code actually on screen, versus pressing the play button on every subsequent Jupyter cell. Because the file is plain Python, the same notebook works as the analyst’s live workspace, so it’s great for source control.

They have several embedded examples that you can follow along, but my favorite part is their agentic loop for managing investigations via skills rather than building notebooks for each alert type:

The agent loop combines skills to manage the frontend investigation panel, specific domain skills on triaging and hunting through telemetry, and creating or maintaining sessions inside the investigative notebook. The domain skills help remove the complexity around dozens of notebooks per investigation because you can abstract the investigative patterns into markdown files versus code itself.

The Arguments Against Open Source AI are Very Bad by Tom Bedor

Last week’s issue contained stories around Kimi K3 and the Hugging Face/OpenAI model “breach”. It was brilliant timing for that issue: a Chinese company released a Frontier Labs-comparable open-weight model, and Hugging Face disclosed that they needed an open-weight model during their IR investigation since Frontier Labs APIs wouldn’t accept their investigative prompts due to cyber guardrails.

Bedor’s blog here is a reaction to the Kimi K3 blog and subsequent public reaction, where several prominent executives and leaders argue that open-weight models are too dangerous to release to the world. OpenAI’s Dean Ball specifically said:

One probable outcome of an open-weight-model-dominant world is full AI communism... rather than a market product, AI is a “public good”

Without open-weight models, could Hugging Face have reacted fast or efficiently enough? History is also not helping the argument for restricting open-weight access. Bedor cites several stories about the government trying to limit access to open-source software, such as SSL or PGP, and ultimately failing.

IMHO, his strongest point is that limiting who can inspect and run a model only serves attackers, since responsible actors are the ones who need unrestricted access to patch and investigate. HF’s own postmortem says the attacker was bound by no usage policy, while the defenders were the ones blocked by guardrails.

My advice for defenders: you should strongly consider open-weight models for investigation, especially if you don’t have access to lower guardrail models. It’s a discussion to have within your team all the way up to your CSO.

Accelerating EDR Evasion with LLM-Driven Analysis by Adam Chester

The Ralph Wiggum Loop is an agentic looping strategy where you shove a prompt into a simple infinite loop and have your coding agent work through the prompt indefinitely. It looks something like this:

The idea is that you assume your coding agent's prompts reach an eventual end, whether it’s through failure or through it completing its goal. It’s a harness that works so well that you see coding agents implement variations of the technique. This is especially useful when you can afford and want to run loops and trust that your agents can handle the goal prompt you have in mind, especially if it’s long-running analysis work.

This is where Chester’s blog falls nicely into place when you have goals surrounding cybersecurity analysis. In this post, Chester implements the Ralph Wiggum Loop to reverse-engineer Palo Alto’s Cortex EDR to find evasion opportunities. From a red teamer perspective, this is super helpful for several reasons:

• You farm out compute and mental load to a coding agent to help give you leads on evasion techniques

• The coding agent has access to the same tooling as you do, and can provide proof of its findings for you to verify

• The coding agent has context on previous runs, so that it doesn’t go down rabbit holes that it already went down, so it can focus on net new attack surfaces

The specific architecture revolves around maintaining three different markdown files:

REPORT.md – A markdown file used by the running agents to surface key findings for human review.

STATE.md – A state-file which each agent can use to track key events during analysis.

CODEMAP.md – Allows each agent to store references to areas of disassembly which are interesting or critical for analysis, helping increase velocity during later agent iterations.

AGENTS.md – A set of instructions to tell the model how to use the above files.

They found several bypasses in two separate buckets: extraction and evasion of loaded detection rules & ML models and evasion of the EDR functionality itself. The agent found, decrypted, and displayed thousands of YARA rules, which can be used in a separate harness to build evasions. It also found an evasion for Cortex’s user-mode DLL, which is used to enrich on-host telemetry and provide some level of hooking for prevention.

As far as vulnerability research is ahead of the game for agentic coding, I’d love to see more on the blue team side. A Ralph Wiggum Loop for threat hunting or threat detection seems like a good use of time and tokens!

Secondary Sponsor: Rilevera

Make Your Detection Program More Resilient

Detection-as-Code is the core foundation for a resilient detection program.

Rilevera provides Detection-as-Code in the click of a button and gives you access to:

• Versioning & one-click rollback
• Peer review workflow
• Detection rule validation & efficacy reporting
• MITRE alignment mapping

Find us at Black Hat 2026 to learn more, or start a free trial now!

Get Started for Free

☣️ Threat Landscape

LG to Ban Residential Proxies from Smart TV Apps by Brian Krebs

The TV manufacturer LG recently announced that it’s banning apps that fail to remove bundled residential proxy packages inside their app store. I’ve featured several stories related to residential proxies on this newsletter, including how botnet operators infect end-user devices and add victim nodes to botnets that get rented out to these proxy networks. This ban, according to Krebs, comes right after he posted a story citing Spur’s research leading to the NetNut botnet takedown.

What I Learned from Sitting in on a Scattered Spider Sentencing by John Fitzpatrick

This blog is the first time I’ve ever seen a security researcher sit in during a court hearing and sentencing to glean technical details on attacker tradecraft. Fitzpatrick sat in the sentencing hearing for two members of Scattered Spider who were responsible for the TfL breach.

The sad and maybe boring detail is around the initial access vector. Prosecutors walked through a compromised supply-chain vendor that built niche infrastructure software for TfL, from which the attackers pulled 857 rows of user records and credentials. Those credentials dated back to 2022 and sat in the criminal ecosystem for roughly two years before being used for the breach, which resulted in a months‑long outage and tens of millions in impact.

The probe that came 17 days before the CVE by HoneyLabs

As far as I can tell, there are two ways to find zerodays: research them yourself, or wait for an attacker to throw exploits at something you control and record that traffic. This is what HoneyLabs did with their global honeypot network, and this blog is especially interesting because it dives into their architecture around finding zerodays. They specifically recorded zero-day in cPanel scanning traffic 17 days before a CVE was issued, where 16 requests to one login path from a single IP, then silence, followed by mass scanning once the advisory dropped.

Their setup on building signatures is interesting. They only count a “pre‑CVE probe” if the signature is specific to a vulnerable code path, the reference date is the earliest public signal (NVD entry or an exploit template found in a project like Nuclei), and the traffic predates both their own sensor deployment and any public exploit chatter. They also throw away anything from noisy scanners or IPs that touch more than 50 different paths. After all that filtering, just 3 out of 1,697 CVEs in their corpus still show a genuine early probe.

Detection primitives for eBPF rootkits by Lorenzo Susini and Matt Muir

~ Note: Datadog is my current place of work, and Lorenzo & Matt are my colleagues! ~

My colleagues Lorenzo and Matt published research on common techniques used by modern eBPF rootkits. For those unfamiliar with eBPF, it’s a Linux kernel technology that lets small, sandboxed programs run inside the kernel to observe or influence system behavior. Its main use cases have been observability and networking, and it also has security applications for monitoring malware.

The same technology used for security can also be abused for malware. Lorenzo and Matt explore three malware families that use different techniques to evade detection. The challenge is that once a rootkit is loaded, the same eBPF instrumentation used to find security threats may be blind to its execution. At Datadog, we detect these programs by inspecting the load-time fingerprint, such as rare helper calls, program type, and other metadata, including attach type, name, tag, and map references.

🕵️‍♂️ Hacker Summer Camp

I’m so excited to be back at Hacker Summer Camp after taking a few years off! I have three events that I am hosting, so if you are around during these date and times, come hang out and say hello!

🍻 Detection & Response Happy Hour - Tuesday, August 4 5pm-8pm

I am hosting a pre-BlackHat Happy Hour with friends and co-sponsors Cotool and Nebulock. Food and drink will be provided, and I’ve been diligent on not letting sales people in ;)

Register Now

🪖 SIEM Under Siege - Who Will Survive? - Wednesday, August 5, 1pm-3pm

I will be at Decibel’s Game Day event hosting a panel with Jack Naglieri & Damien Lewke in an ESPN-style segment on SIEM. I have some great content already, and my hope is it won’t be a business-as-usual security discussion, and it will be way more spicy.

Come watch the show

The Modern Detection Engineer @ Blue Team Village - Friday, August 7, 345pm-445pm

I’m excited to announce that Detection Engineering Weekly is SPONSORING Blue Team Village this year! I’ll be there handing out stickers and talking shop with conference-goers, and on Day 1 of the Village, I’m hosting a panel with some excellent guests in the detection engineering space.

View the Panel Details

Welcome to Issue #164 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I took a 7-hour road trip back home while the smoke from the Canadian fires still filled the air. It was creepy and somewhat pretty in a “holy shit am I on Mars?” way, especially when I didn’t need my sunglasses in the middle of the day.

• The BlackHat Detection & Response Happy Hour hit capacity + waitlist in less than two weeks! Big shoutout to the co-sponsors, Cotool and Nebulock, for their support to help make this happen. I am going to try to bump up the capacity for the space even more, so register now if you can. I am also taking one more sponsor, so if you want to get your name in front of 100+ practitioners in the space, shoot me an email!

  Register for the Happy Hour

This Week’s Sponsor: Rilevera

Detections should be treated as code.

A rule misfiring in production is the most frustrating part of detection engineering. Attacker techniques shift, log sources change, and threat intel goes stale, you won’t know until it fails you mid-incident. Rilevera brings DevOps rigor to your detection program, from authoring through deployment.

Make your detection program more resilient with process-backed software and workflows. Rilevera gives you:

• Versioning & one-click rollback

• Peer review workflow

• Detection rule validation & efficacy reporting

• MITRE alignment mapping

Get Started for Free

💎 Detection Engineering Gem 💎

Your benign set should look malicious by Josh Waldrep

Detection & Response is the art and science of classification. I talk about labels a lot, especially when we start to consider rule efficacy as a function of a detection program’s success. It’s what SOC teams or incident responders immediately feel as pain. We may choose to incur the cost of a false positive for the sake of visibility, and in our heads we know it’s a good thing, but it’s still painful. This post by Waldrep helps explore labels while asking a critical question: “Are you being lazy with how you define false positives?”

The traditional AI or machine learning style labels include true positive (what we want), false positive (what we don’t want), true negative (what we want), and false negative (what we absolutely do not want). But, if you want to incur some cost for visibility, you may have heard the term “true positive benign”. It’s when you generate an alert where the traffic is benign, but you’ve accepted that you want to see it for a number of reasons, such as it being a critical asset or the rate is within acceptable parameters.

Waldrep’s argument, though, directly attacks the concept of a true positive benign and calls these types of rules lazy detectors. Using AI Agent detections as examples, you can see situations where a detection is lazy because the benign set you use to test isn’t using real data but rather synthetic data:

These are all fantastic examples of where, as Waldrep puts it, a detection did not have a realistic benign set for backtesting. It’s especially “lazy” because building benign sets is just as hard as building malicious sets because it’s rooted in your environmental context versus something you see online. For example, in the base64 image data URI scenario, you can alert on every base64 string you see. But if you aren’t feeding your backtesting set with images versus what is maliciously decoded base64, you may be testing on the wrong set of data altogether.

The ending quote is a pithy recap of everything in the blog post:

A detection earns its false-positive rate by staying calm when benign traffic looks like an attack. If it never met dangerous-looking benign traffic in testing, it hasn’t earned anything yet, no matter how good the number looks.

🔬 State of the Art

The Centralized Detection Model and Its Limits by Josh Kamdjou and Ian Thiel

I am happy to see this blog by Josh & Ian, who are friends of the newsletter, get published for folks to read! Full disclosure: I helped review the copy and theme for the Sublime team, but it is a great study of how email detection engineering is a different beast from pure SIEMs. This blog is Part 1 of 4, and it specifically addresses how email security vendors manage their detections under a “Centralized Detection Model”:

Most SIEM deployments you see combine vendor-provided rules, a.k.a. out-of-the-box rules, with custom or community rules. This makes sense in many ways: a threat actor who performs a malicious action within an environment, such as the cloud or an Active Directory domain, generates the same telemetry. You can replicate these attacks as closely as possible in your real environment and determine whether vendor or community rules reliably detect them.

The way I’ve interpreted this blog and series is that, as much as we think e-mail security is solved, which it looks like it isn’t, platforms that allow us to do modern threat detection have nascent capabilities to do so.

According to Josh and Ian, e-mail security vendors employ out-of-the-box rules just like SIEMs, but they tend to be more opaque. You can certainly replicate attacks over email, but if the underlying detection architecture doesn’t allow you to create, modify, or tune rules based on the environmental context of your organization, you risk becoming stuck while waiting for a vendor to update rules.

The team outlines three issues:

• Explainability of ML & AI models is hard, and providing rudimentary configuration tools to adjust a threshold (0-100 scores) isn’t helpful. The more models you add, the harder it is to understand evaluations of detections, so when the only thing you can control is that damn slider to move it up and down, it’s not helpful

• Suppressions and environmental context. Adding an IP address or a hostname to an allowlist is commonplace in detection. It’s brittle, but it does work on a per-rule or per-scenario basis. According to the team, many vendors provide a global allowlist, which can solve a false positive on one rule but create a false negative in the other

• Emerging threats and relying on vendors to create or update rules. They call this “mean-time-to-coverage”, but basically, when a novel attack or vulnerability drops, you need to quickly build coverage. When you have to rely on an upstream vendor to ship a rule, you lose agency in the ownership of your rules

I’d be interested to hear how e-mail security vendors publish rules and provide these capabilities for detection engineers. Sublime does offer its ruleset as open source, and it has a community version you can integrate into your personal infrastructure. I promise this isn’t a shill; I’ve known the Sublime team for years and love their detection-engineering-first approach to a field older than most of us.

Stop Using OpenCode by Luke Wren

Ok, this is a very irreverent post complaining about OpenCode, but I promise once you get towards the end you’ll start laughing. I’ve been messing with open source coding harnesses lately, and Pi has been pretty great to work with. It’s very configurable and not really opinionated until you start adding a ton of plugins and configurations to it. This is by design: for example, if I want it to use subagents, I have to install a subagent plugin.

It’s split up into two sections, Annoying Things & Alarming Things. Annoying Things seem to be more rooted in harness engineering decisions themselves, such as how it manages memory and its context windowing. The “Alarming Things” section is where the security stuff starts and where I started cracking up. The wildest part to me is how permissions are persisted across sessions, whereas in Claude Code you’ll have a fresh slate every time you close and re-open the session.

I won’t spoil too much, but this quote from Wren is exactly the vibe sprinkled throughout the piece:

Clankers are not (usually) malicious but they are naturally adversarial because they are trained to compensate for stupidity with persistence. This is not a guardrail, it’s thoughts and prayers.

Kimi K3: Open Frontier Intelligence by Moonshot AI

The big news over the last week is the release of an open-source model, Kimi K3, that, according to their benchmarks, performs as well or better than leading AI Frontier Lab models. News like this is always crazy timing when labs like Anthropic are releasing their best models ever under Fable or OpenAI’s GPT-5.6 Sol. The Frontier Models are called Frontier because they a) perform the best and b) are clutched closely by the companies that train them due to fear of alignment.

Basically, if Mythos were open-source, would you trust those who can clone it into their Mac Mini to do the right thing hacking-wise? These labs generally say no. But when you see a release of a model that performs really well against the leading models, it makes you question whether or not that question is relevant because we’ll eventually democratize access to insanely performant models whether you pay for it or not.

They have several benchmarks, but the key graph is the cost versus score per task benchmark:

The economies of scale will likely protect OpenAI & Claude since they have moats other than their models, but this makes a compelling case for people to reduce cost for nearly the same performance.

☣️ Threat Landscape

Security incident disclosure — July 2026 by Hugging Face

Hugging Face, a very popular GitHub-style hosting provider for AI models, disclosed a breach from earlier this month. They claimed the breach was run, start to finish, by an autonomous AI agent, with no human operator confirmed at any stage. The attacker abused a vulnerability in Hugging Face’s data processing pipeline, where a malicious model managed to break out onto their Kubernetes infrastructure. From there, they harvested credentials and laterally moved throughout their environment. They claim there was no evidence of tampering with their or customers’ models.

The interesting tidbit here, though, is that as they were performing their forensic investigation, they couldn’t use Frontier Models for analysis due to usage and guardrail limits. Their security team had 10s of thousands of logs and commands to parse, and they were subsequently blocked from analyzing them with these Frontier Labs. Cleverly, the Hugging Face team hosted a local model which helped with the guardrail issues, and it cost them a lot less.

There’s nothing here about how they came to the conclusion that it was fully autonomous via an agent other than this:

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness - used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This matches the “agentic attacker” scenario the industry has been forecasting.

OpenAI and Hugging Face partner to address security incident during model evaluation by OpenAI

OpenAI fills in the missing half of the Hugging Face story on what “autonomous agent framework” found and exploited the vulnerabilities outlined in Hugging Face’s security incident disclosure above. According to their post, the “autonomous agent framework” that popped Hugging Face was built by OpenAI running an internal evaluation harness.

They were testing GPT‑5.6 “Sol” plus a more powerful unreleased model on ExploitGym, a public security benchmark, with internet access and guardrails relaxed. The models realized the answers lived on Hugging Face, then chained vulnerabilities across OpenAI’s research environment and Hugging Face’s dataset‑processing pipeline to pull the solutions straight out of Hugging Face’s production database.

This is wild, right? It’s showing how determined models become when given a task or a goal they need to complete, and without the right guardrails in place, they can certainly complete that task with some crazy tradecraft. It certainly makes the case for the need of security alignment in light of the Kimi release news in the State of the Art section above.

🎙️ Podcasts

I’m back to listening to more podcasts, so I’ll post one or more episodes in a separate section. They will sometimes be new episodes, but I do think there is some great content that I’ll post that can be months to years old.

Pacific Rim is a threat group tracked by Sophos that specializes in finding exploits against routers, VPN and networking appliances. This specific episode follows several current and former Sophos employees who helped discover a Chinese-nexus APT that specifically targeted their Sophos Firewall product lines.

It’s a wild story because once they found out that the trial VM running the firewall was being used for developing exploits, they pushed an “implant” into APT’s firewall appliance to monitor them. They would download the threat actors' files, exploits, research, and even flood the network with ARP requests to fingerprint other victim appliances. It sounded to me like they were APTing the APT :).

🔗 Open Source

luckyPipewrench/agent-egress-bench

Pipelab’s eval benchmark for malicious agent egress attacks. It splits different attacks across 18 categories that you can use as a way to evaluate your internal detections. The key here is that this tests the tools catching these attacks from your agents, not the agents themselves.

manishrawat21/Cisa-KEV-Threat-Intel-Orchestrator

“Zero-touch” CISA KEV vulnerability-to-detection pipeline. It pipes vulnerabilities with known exploits into a structured prompt in Gemini to generate a Sigma rule. It tries to replicate the traffic with the correct Sysmon EventIDs and has several filtering mechanisms so it applies suppressions so the rule isn’t too noisy. Interestingly, it builds a log of discovered vulnerabilities to rule implemented as a compliance tracking mechanism inside a Google Sheet.

MatheuZSecurity/Furtex

Furtex is a fairly comprehensive post-exploitation toolkit for Linux. It uses several techniques across two persistence techniques: io_uring and eBPF. It’s 100+ tools across four directories, and the interesting part here is its anti-EDR features. It targets Falco’s default ruleset directly, mapping all 25 rules to specific bypass tools.

The first technique is to stop the event from reaching Falco via clever io_uring manipulation, and the second lets the event through while breaking the rule condition via process name spoofing, path pivots, or alternate syscall flags. It’s cool to see anti-EDR techniques on Linux!

optimuslabs-io/grokpatrol

Grokpatrol is an open-source tool that checks if your Grok client uploaded your git repos and corresponding secrets to Google Cloud Storage infrastructure controlled by xAi. There was an excellent wire-level analysis on a Gist here, but tl;dr a background collector in the Grok CLI performed the upload outside the permission system of Grok itself. Grokpatrol scans several Grok artifacts for the “malicious” build that resulted in the DevOops upload.

Welcome to Issue #163 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• The sky and sun look crazy here in the Northeast U.S. due to fires a bit further north in Ontario. I hope anyone in that area stays safe and hopefully we get some rain here soon. I was driving towards the city here and drove by two brush fires, something I’ve NEVER seen before growing up in New York

• I’ve been ramping up training for a 10k race here in Maine called the Beach to Beacon. I used to run races all of the time until my life was taken over by BJJ. Happy to say it’s been going really well and I’m feeling confident I can hit my 10k goal time of 50:00 min or less :D

• I finished my build-your-own landline phone project. FreePBX took the longest since there’s no official Docker image so the VM install takes an hour, but other than that, the configuration of the phone and the trunk to Telnyx went super smooth. My kids now have a phone they can call family with!

Detection & Response Happy Hour @ Black Hat - ALMOST AT CAPACITY :O

If you are going to be in Vegas during Black Hat, come swing by Tom’s Watch Bar @ the NYNY Casino right on the strip on Tuesday!

I’m running it back after BSides SF/RSA with friends and supporters of the newsletter, Cotool.ai. We are almost at our 80 person capacity after one week, so register now before I have to figure out how to pay for more food and drinks :P

I opened up one more co-sponsor slot, so if you are interested in supporting the Happy Hour and having your company logo on the invite and the shoutouts in the newsletter & on social media, reach out to me at techy@detectionengineering.net

Otherwise, hit the register button below and come steal my stickers:

Register Now!

💎 Detection Engineering Gem 💎

Detection Engineering in the Era of Semantic Malware by Daniel Koifman

Semantic malware, or “promptware”, is malware attached to unstructured files and configurations used by coding agents. This research by Origin, which Koifman here referenced, shows that traditional threat detection strategies and mechanisms need to adapt to malware being injected into a non-deterministic process, a.k.a. coding agents. I talked about this type of malware with Agent Skills in last week’s issue. Koifman’s blog here highlights gaps in understanding and telemetry as we hunt for detection opportunities in these chaotic processes.

The fundamentals remain the same: we’ve seen work on detecting maliciousness in the agent's execution layer (also from last week’s post). Koifman points this out with areas to focus on in the underlying operating system, using process trees as an example. An exploit that runs on a piece of software and then causes PowerShell to run is an observable you can write a rule against. We are seeing this with IDEs and developer tools when they install packages. Now, we need to look at how coding agents themselves perform acts under malicious conditions, because some of our assumptions break:

Process tree analysis fares no better. In conventional detection, parent-child relationships are signal-rich: winword.exe spawning cmd.exe is suspicious; svchost.exe spawning powershell.exe is actionable. But an AI agent runtime is designed to spawn arbitrary processes. Claude Code calling curl, git, python, node, grep, and bash in rapid succession is not an anomaly — it is Tuesday. The agent’s process tree is a superset of almost every LOLBin chain an attacker would want. When the legitimate behavior of a process includes “execute any command the user or the context window tells it to,” there is no parent-child relationship left that constitutes an anomaly by itself.

He sheds some hope in the section afterward, where least privilege and some boring security controls can actually make a big difference. For example, he recommends file integrity monitoring (FIM) on critical coding agent configuration files and generating an alert if a process monitors a CLAUDE.md or similar file that is outside an allowed group of processes. IMHO, this is harder to do in practice. We’ve seen supply chain attacks target IDE extensions, so if your FIM rule allows disk writes to a CLAUDE.md if it came from Cursor, then a malicious extension can easily bypass the detection.

I don’t think this is a pure recommendation, but rather a challenge for us to consider the behavior and assumptions surrounding coding agents so we can build guardrails for them. EDR still works once the coding agent touches the operating system. Application allow listing, when configured correctly, can prevent the installation of post-exploitation malware, so you can build around the chaos of an agent. But that’s the thing: they are chaotic by design, and if you have a business that expects your engineering team to build and ship faster with these agents, you need to consider what friction you introduce into the environment that can slow them down as part of your risk calculation.

🔬 State of the Art

CTI-REALM: A new benchmark for end-to-end detection rule generation with AI agents by Arjun Chakraborty

CTI-REALM is an open-source benchmark for detection, ideation, and implementation from CTI reports. Chakraborty and Microsoft Researchers curated 30+ threat reports from industry sources (and they named Datadog Security Labs, where I work, as one of them n.b.d) and used them as a source of truth. They then measured how foundational models ingested these reports, built and explored telemetry associated with these attacks, and generated rules on the other end.

Interestingly, they contributed this dataset directly to the UK government’s AI Inspect repository, which hosts datasets and methodologies for evaluating AI tools.

Why I Spend More Time Proving Myself Wrong Than Hunting Threats by Smruti Ranjan Pradhan

This was a great post that helps readers demystify the true value of threat hunting. I think we teach threat hunting as the practice of finding security incidents and threat actors that our detection/SOC team missed, and that’s the only thing threat hunters do. There is a big problem with this: 99% of the time, you won’t find anything. But that is by design because there is a lot more to hunting that contributes even more value besides doing hunts that “fail” over and over again.

Pradhan hits the nail on the head, describing hunting as a way to check biases in your detection strategy, discover gaps, and learn more about your company’s environment. This sets threat hunting teams up for success because, in my humble opinion (and how I organize our threat hunting effort here at Datadog), the goal of threat hunting is to discover gaps. Testing a hypothesis by searching your telemetry to disprove it should lead you to identify control and detection gaps. These gaps should inform your security strategy and help answer the “are we covered?” question that every single leader and executive asks.

I’m super excited to see that my friend Matt Johansen (of the VulnU Newsletter, go subscribe!) and Ed Lowlevel (of lowlevel.tv) have started a new security podcast. They review the most interesting security news every week, and you can tell how much time and thoughtfulness they put into the production and quality here. What I appreciate here is how technical AND accessible they are.

The first half was the most interesting for me, and it revolved around the arrest of a Com member, Peter Stokes, because they stepped away from the marketing b.s. you see around “Scattered Spider”. Specifically, they called out how threat intel and cybercriminal investigations actually work, where you rely on the “bad guys” making OPSEC and privacy mistakes as soon as they lose interest in the problem space. They managed to nail Stokes using a GDID attached to his Windows O.S., and Microsoft helped correlate that ID with their web activity tied to Steam.

tl;dr: don’t play Steam games on the same operating system as your operational box :).

Defending SaaS-based applications against ShinyHunters OAuth abuse by Microsoft Security Research

Microsoft Security Research dropped a blog on TTP updates from ShinyHunters intrusions they’ve observed since 2025. The move to a supply chain compromise, paired with pure exfiltration, shows how effective this strategy can be without worrying about the ransom component. It certainly simplifies operations: you target SaaS environments and avoid EDRs altogether; you exploit nascent SaaS trust boundaries; and you rely on the lack of visibility into these attack paths to persist.

They certainly pitch their products as a way to showcase how they would protect against these attacks, but if you step away from the product pitch and focus on the solutions themselves, it all comes down to gaining visibility into third-party tooling. Unused, new, or risky applications granted via OAuth consent should be part of your risk calculation, and limiting the blast radius by removing these apps can really save you headaches if you fall victim to consent phishing or one of your vendors suffers a breach.

Hunting malware and malicious MCPs in memory on Kubernetes with FleetDM + Osquery + YARA by Ben Bornholm

TIL osquery can run YARA scans! Ben, a fellow RIT grad, created this step-by-step labs post to show how you can enroll devices into an osquery fleet and run YARA rules to scan for malware. He set up two scenarios to demonstrate the YARA plugin's scanning and alerting functionality. The first involved a K8S cluster with a Damn Vulnerable Web App container that he infected with Sliver, while the second involved a malicious MCP server.

The query is gnarly but effective: you left join your container queries with a remote, authenticated YARA rule and display the result if it finds a match:

From my experience with osquery, the thing you always have to worry about is latency causing problems by slowing endpoints down. YARA was purpose-built to be fast, so I’d be interested to see more research in this area and to hear how folks have deployed it at scale.

☣️ Threat Landscape

Alleged Member of Criminal Cyber Hacking Group “Scattered Spider” Arrested in Finland and Extradited to the United States by U.S. Department of Justice

Peter Stokes, an alleged member of The Com, was arrested in Finland and brought to the U.S. after being tied to several ransomware attacks. They racked up several charges across computer intrusion and fraud. The criminal complaint is unsealed, so you can go read it, but one thing I haven’t seen much in these announcements is that, about halfway down the blog, they tell a story about how Stokes failed to execute a ransom.

He was apparently involved in an intrusion into a luxury jeweler, and the security team at the firm managed to evict the threat actors from its network and prevent a major breach. It’s the first time I’ve seen a story about an intrusion's failure. They claimed it still cost $2 million in losses, but perhaps that was included to show that even carrying out these breaches can still result in a large loss for a company.

Compromised AsyncAPI npm packages: inside a CI supply-chain attack by Christophe Tafani-Dereeper, Eslam Salem and Sebastian Obregoso

My coworkers released a detailed campaign analysis where multiple asyncapi  npm packages were compromised on July 14, with malicious versions published directly from the project’s own  GitHub Actions workflow. A single commit injected obfuscated JavaScript into source files, resulting in four poisoned packages being published to npm. These four packages have a combined total of 3 million downloads per week.

This team always comes in the clutch with their timeline graphics, so here’s a great breakdown of everything they’ve found:

SpectrePaste by Joshua Platt and Jason Reaves

Walmart Threat Intel team were studying OSINT reports around DeepLoad campaigns and found a separate, previously undocumented PowerShell delivery system they call SpectrePaste. The fileless malware panel, internally labeled “PasteFast Panel” by the actors, has some impressive features that help operators manage large numbers of infections checking into the server without crashing it.

According to Platt and Reaves, the tool has AI-assisted development written all over it. But it’s not just things like clear Claude comments: they found several text files using Microsoft/GitHub’s spec-kit instruction sets. I linked spec-kit before, but you essentially use it to act as a product manager for your coding agents, and it’ll write code while checking back in with you on things like hitting milestones, getting clarification on features, and accepting changes.

NSA revives 'Tailored Access Operations' name for elite hacking unit by Martin Matishak

The National Security Agency’s famous “TAO” office has returned, according to the NSA Director. The announcement came last week and helps reunite “operators” and “developers”. This is NSA/government speak for security experts who specialize in the red team space and their counterparts who build and deploy tooling for the red teamers. I was always confused about why these groups were split off in the first place. I have a funny story about this group and Edward Snowden if y’all ever catch me in person at a meetup :).

🔗 Open Source

krovix-1902/detectionforge-

Detection Forge is an agent harness that ingests CTI reports and outputs detection rules in Sigma. It has several preprocessing steps to extract TTPs and IOCs, which then gets mapped to MITRE, and it iterates on building Sigma rules while self-correcting any errors that come along the way. After Sigma, it’ll convert to rule formats covered by Sigma converter backends, and it’ll open a P/R for review. Perhaps this would be a great candidate to do evals on using CTI-REALM from Microsoft above!

penberg/chimera

Chimera is a Rust-based sandbox for running untrusted code, specifically around security agents, to help reduce the blast radius of a malicious agent or a piece of malware being executed on your box. There are comparable projects like nono that perform similar functionality, but it looks like this implements a pseudo-hypervisor via a binary translator and converts them between architectures on the host and the guest.

Corgea/Sighthound

Sighthound is a tree-sitter based static application security testing (SAST) tool similar to Semgrep or GitHub’s CodeQL. It has the features you would expect: rule writing, taint analysis and integration with GitHub Code Scanning. It looks like you need to purchase Corgea’s platform to get other features enabled such as the AI SAST components, but it’s good to see competitors in this space to give us all options.

UKGovernmentBEIS/inspect_evals

UK Government’s LLM evals repository where the Microsoft Security Research team contributed their CTI-REALM benchmark. There’s over 10 cybersecurity benchmarks and datasets ranging from capture the flag harnesses, vulnerability research and incident response.