Welcome to Issue #162 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I had an excellent long weekend celebrating the 4th of July here in the U.S.! It was a good lead-up to a rather disappointing World Cup loss on Monday :(

• We recently bought a kids’ WiFi landline phone thing so they can call family and chit-chat. Let me tell you: it’s been terrible. The quality/service is poor, and it just feels cheap. So, I’m trying my hand at rolling my own FreePBX server with an upstream trunk provider. I haven’t been this excited about a project in a long time :D will report back once I get it deployed

Detection & Response Happy Hour @ Black Hat

If you are going to be in Vegas during Black Hat, come swing by Tom’s Watch Bar @ the NYNY Casino right on the strip on Tuesday!

I’m running it back after BSides/SFRSA with friends and supporters of the newsletter, Cotool.ai. It was super chill at RSA with no vendor b.s., so escape Mandalay Bay and come talk shop with other practitioners.

Register Now!

💎 Detection Engineering Gem 💎

End-to-end detection validation using coding agents by Kyrre Wahl Kongsgård

This blog is one of the single best deep dives on detection validation I’ve seen in years. It hits a bunch of themes, including types of detection testing and architectural decisions for building and deploying end-to-end tests, and clearly describes an elegant and repeatable agentic loop for this use case. Let’s break it down piece by piece, because there’s a ton here and I highly recommend reading this one if you don’t want to touch any other story in this issue.

Security folks tend to steal concepts from SRE and developers and relabel them with fancier, cooler names, but the underlying principles remain the same. The picture above from Kongsgård shows how we celebrate concepts like testing and “chaos engineering” in security and map them to the security telemetry lifecycle. Regression tests, for example, focus only on verifying that an input (telemetry) produces an output (alert). Synthetic ingestion is an integration test of the ingestion and shaping of telemetry to generate an alert. End-to-end testing looks at the full telemetry → detection → response pipeline.

My favorite themes

Since this post has enough content to fill several posts, I’m going to point out two of my favorite themes so it doesn’t feel like I’m repeating or rewriting Kongsgård’s content.

Lab environment and TTP framework

Booting labs up to run simulations takes a ton of time and effort, especially if you are starting from scratch. The goal is to replicate your environment as closely as possible, but there are always trade-offs in simulation. Some of these tradeoffs include:

• Environment mirroring: A host running in a VPC can help mirror what your endpoints or cloud resources look like, but it won’t be exact. Detonating potentially dangerous tooling inside a production environment can introduce externalities or even real security incidents if you aren’t careful

• Baseline activity: A user, much like an agent, is non-deterministic. The telemetry they generate from normal activity is just as important to model as the malicious traffic itself

• Provisioning discipline: Running a small amount of Atomic or Stratus Red Team tests is manageable from an individual detection engineer’s perspective. If you want to run your whole catalog of detections, you need to start thinking like an SRE or software engineer, as you’ll hit scaling and drift issues with your infrastructure

Kongsgård’s detonation environment has a high level of discipline to address these tradeoffs. The section on the Lab environment uses several DevOps paradigms, such as golden images, configuration management, and deployments via GitHub Action runners. Under the hood, they use Meta’s TTPForge as their adversary simulation framework for execution on detonation hosts. It offers a content-rich, multi-step attack-generation feature set that is adaptable to their agent-harness framework.

Agents as validation drivers and the schema knowledge base

Singular prompt, one-shot agents have their place in implementing agentic systems in security, but they tend to perform poorly as tasks become more heterogeneous. Since the task is end-to-end detection validation with TTP generation, rule tuning, and detonation, a Claude Code or Codex agent would not be sufficient.

The harness is the differentiator for anything agentic security. Remember that! Their agent isn’t improvising an attack; it’s following a plugin that teaches it to write TTPForge YAML files, ship them over SSH to a lab host, run the detonations, and then queries Splunk to see whether telemetry arrived and the detection matched. When a step fails, lifecycle hooks block progress until the agent finds the issue. Here’s the high-level architecture:

My recommendation to all my readers is to design your agentic workflows around single agents that do one thing very well. It’s as if you are extracting one piece of expertise from your brain and encoding it into a prompt to do a single thing. In this particular case, Kongsgård designed two plugins to perform discrete tasks.

• detectionkit builds the TTP definition via TTPForge, writes the detonation test, deploys and runs the detonation. Its singular purpose is to replicate threat actor activity in a common & repeatable lexicon

• splunk is the plugin that performs the validation that the correct telemetry was captured, the search for the rule was performed quickly, and continuously discovers index structure to understand rule performance and drift

Security vendors who sell agentic capabilities typically don’t expose their harnesses at the level of detail shown in this blog. Researchers and open-source enthusiasts are quickly catching up with these vendor-led harnesses, and this truly gives detection teams agency to choose between build and buy.

It’s been easier to admit that I can’t imagine a world without a Claude Code or Codex. The prompt was the star of the show for the first year or so of this coding agent frenzy, but it’s now squarely the quality of the harness that brings detection to the next level.

🔬 State of the Art

Detecting Agentic Threats in Claude: Writing Rules on the Execution Layer by Andrew Byford

This is a Part 2 post from Byford’s previously featured work on writing rules for Anthropic’s Compliance API. The cool part here is that, unlike his last post, which focused on the prompt content itself, this looks at the execution layer of the coding agents. I’ve always interpreted the execution layer as how the agent interacts with the filesystem itself. This presents unique detection challenges because, in my opinion, the impact is the same, such as downloading and executing a binary, but the paths are different, such as malicious skills, reading a malicious prompt, or loading a malicious plugin.

Byford splits the threat categories into five distinct buckets: excessive agency and permissions, supply chain threats, dangerous actions, sensitive information disclosure, and data poisoning. The architecture is clever where the Compliance API is used as an enrichment backdrop during investigations, so you can combine unstructured data from prompts with the structured data generated from Claude hooks:

And here’s the enrichment layer after a SIEM rule fires from the OTel collector:

Much like detection engineers have had to become supply chain security experts in the last two years, I don’t see a world where we also must become AI Coding agent experts in the next year or so. I never considered using prompt and response content generated from coding agents in the Compliance API as additional context for SIEM alerts, so I’m now going to steal that idea and see what I can do at my day job (sorry, Andrew!).

SOC Bench by DeepTempo

Evaluation datasets are critical for understanding model performance. Much like in my analysis of this week’s Gem, one-shot prompts can perform well under very constrained conditions, but without something to measure real-world malicious vs. real-world benign, you should limit your confidence in virtually all agentic security applications.

I found this SOCBench website & corresponding open-source repository, and it reminded me of Cotool’s Research benchmarks with similar datasets. This specific one includes a NetFlow dataset containing both malicious and benign network traffic. It’s also a bit more opinionated about persona benchmarks, ranging from SOC analyst to detection engineer, and includes more architecture, with tool catalogs and playbooks for those personas. Their first benchmark around detecting maliciousness:

Anthropic performed the best but it looks like it cost the most. I find it interesting that OpenAI’s benchmark had the threat analyst perform the best vs the SOC analyst in the other two.

Skills Registry Threat Models by Andrew Nesbitt

Two issues ago, I linked a blog by Aman Khurana that helped demystify the peculiar supply chain architecture behind VSCode extensions. The big takeaway I took from that blog is that not surprisingly, the more security engineers dig into supply chain security, the more they realize how difficult it is to piece together OSS ecosystems to perform effective detection and blocking. Coding agents are built to be autonomous and extensible, just like OSS. The difference lies in the non-deterministic way these agents perform coding tasks, due to intentionally designed boundaries.

In this post, Nesbitt unveils his threat model around coding agent skills. A skill is a bundle of prompts, code, dependencies, and tool permissions. Anytime a skill is used, the skill prompt is injected into the context window, and a set of tools and scripts gets exposed to the coding agent. The more frightening part of the Skills supply chain security is that, instead of a single npm command installing other packages in Node that eventually land a piece of malware, you can have a Skill install packages from virtually any ecosystem, and sometimes those packages are just more prompts.

I don’t think we are truly ready for a large-scale malicious Skill campaign, much like what we’ve seen with the likes of TeamPCP. Nesbitt points out several issues with how Skills are installed, deployed, and managed, and it certainly seems that this ecosystem is in the same stage that npm was in several years ago.

☣️ Threat Landscape

FBI Seizes NetNut Proxy Platform, Popa Botnet by Brian Krebs

The DoJ nabbed another residential proxy platform linked to the Popa Botnet. Krebs post here helps aggregate some of the data published by researchers at Google, Lumen and Spur. The wild part to me is that this proxy platform is linked to an Israeli company, and I’ve always assumed that these networks are owned by non-Western firms who are harder to work with outside of the U.S.’ sphere of influence.

These types of botnets finally figured out how to monetize without DDoSing. Krebs referenced research from Spur that nearly 50% of TV Apps on the LG Smart TV platform add the TV to these botnets, which are then sold as residential proxies.

I found a malware hiding in my tailwindcss config file. by Couch Potato

Super interesting write-up from a developer who encountered a Contagious Interview-style backdoor in their Tailwind configuration. They never figured out how it got there, but the indicators are classic Contagious Interview:

• Targeting developers and backdooring their code

• C2 server communication to an immutable blockchain style API

• Rewriting git history to conceal the compromise

It didn’t necessarily say what the impact was or whether the campaign resulted in data exfiltration. They did find several unknown processes running in their production environment, so likely something happened there. If I had to guess, it was a PwnRequest due to the rewriting of the git history, but that’s about as far as I’ll go before I start placing bets.

Linux Backdoor Targeting iKuai Routers by dmpdump

This is a cool Linux backdoor writeup of a piece of malware that, based on my ~limited research, targets a Chinese-focused router typically deployed to East Asian/Chinese businesses. It’s an ELF binary that impersonates OpenWRT’s libjson_script.so.0. It was hard to ascertain at first, but it certainly is not a shared library and runs in userland. I don’t necessarily know whether the victimology is Chinese firms, which could make this a Western-based piece of malware, but it seems compact and very specifically designed for one router brand, which makes it smell like an APT implant.

ARToken: Inside an EvilTokens affiliate panel targeting Microsoft 365 by Michael Kelley

The TALOS research team uncovered an offshoot of EvilTokens, a device-code phishing-as-a-service kit. Kelley uncovers the initial BEC-style lure and then reverse-engineers the kit to find modern front-end components, such as single-page application lures, and a full backend dashboard written in React. Some of the differentiating features of ARToken Kelley found include keyword searching across victim mailboxes, post-exploitation tooling against victim SharePoint servers, and even collaborative session links for operators working on the same ARToken deployment server.

Lydia Graslie’s Gem from last week helps protect against some of these attacks, especially if you monitor which Microsoft management surfaces emit audit data for device‑code flows and token lifecycles, and treat gaps and schema shifts as first‑class detection problems.

🔗 Open Source

DeepTempo/socbench

SOCBench’s open-source harness for evaluating alert datasets. The current dataset only contains NetFlow telemetry, but it looks like they want to add more. Their harness is the most interesting between playbooks, personas and how they run the evals themselves.

secdev02/EasyTokens

EasyTokens is a device code phishing toolset that emulates device code phishing as a service kit like EvilTokens. This one is more focused on performing the device code phishing attack itself, so you can use this to pivot into cloud and M365 environments.

kernelstub/Nox

Nox is an open-source attack surface scanning tool. There are 300 module plugins across 24 different categories. You can run each module individually or run a full scan that steps across all 24 categories to find everything from exposed credentials, vulnerabilities and OSINT findings.

phishdestroy/shortdot-evidence

Shortdot is a registry operator that hosts seven top-level zones (TLDs) that, according to PhishDestroy, almost exclusively contain fraud, phishing, and malware websites. This repository holds all of their research, enumerating the zones and their phishing website analysis across the seven sketchy-looking TLDs. Their research also includes financial research and how Shortdot charges ICANN fees to operate these zones.

Welcome to Issue #161 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I had an excellent vacation at the beach with my family! We stayed at an Airbnb with a 1-minute walk to the ocean. There’s something about the crashing waves and the smell of salty water that makes me wish I could afford a house there D:

• I am locked in & going to BSides LV/BlackHat/DEFCON! I’ll be posting details for my Detection & Response Happy Hour next week with sign-ups. Mark your calendars for Tuesday, Aug 4 at 5 pm :)

• I opened sponsorship slots up for the summer, so if you’d like to work with me on ad placements or opportunities to work with the Detection Engineering Weekly brand, shoot me an email: techy@detectionengineering.net

• Lastly, I opened a content submission page for folks who want to get their research and blogs in my reading queue. It’s much easier for me to use this then accidentally miss something on social media, Slack or e-mail!

  Submit a blog

💎 Detection Engineering Gem 💎

Control Plane

Where Continuous Access Evaluation Stops Being Continuous

Not every identity attack ends in a stolen token, but increasingly many do. Device-code phishing, adversary-in-the-middle proxies, OAuth consent grants: the front ends look nothing alike, yet they tend to converge on the same prize, a legitimate access or refresh token that Entra issued after a sign-in it had no reason to question. Sometimes that sign-i…

Read more

21 days ago · Lydia Graslie

Defense-in-depth is an overused phrase in security marketing, but it’s one of the few “buzzwords” where the definition matches in marketing-speak with what it means in security operations. At its core, detection & response is a hedge against when security controls fail. Examples of this include someone entering their username, password, and security code on a phishing page, or someone downloading an infostealer binary from an allow-listed domain, such as a CDN, and running it. The important part here is that detection engineers identify the attack paths that threat actors take when those controls fail.

Lydia’s blog (hi Lydia!) is a great example of the nuances of a powerful security control, Entra’s Continuous Access Evaluation (CAE), and how even the perfect implementation of that control can fail. Both infostealers and attacker-in-the-middle phishing pages are regularly stealing access tokens from victims, and when these tokens get into the hands of threat actors, they can use them to pivot into a production Entra environment. Microsoft implemented CAE to help combat long-lived tokens through a challenge/response mechanism to catch stolen tokens:

The idea is that legitimate or malicious access tokens should be evaluated against access policies and controls, and Entra can catch a stolen access token before a threat actor interacts with the target environment. It’s an excellent security control that is now the default for Entra environments, but much like multi-factor authentication, it has its sharp edges:

• There’s a 1-hour expiration window when the issuing client does not have a CAE-enabled auth flow

• Resources that don’t have CAE can still be interacted with, meaning a bypass of a CAE-enabled client against a non-CAE-enabled resource is possible

• IP restrictions can revoke the key quickly, but infostealers and phishing kits help provide geolocation and IP information, which can help bypass this restriction

Lydia provides a helpful coverage map for when each control fails and what you can do to “hedge” against a stolen token. This is where telemetry on hosts and cloud resources, combined with identity telemetry, provides a much stronger defense-in-depth approach when the best security controls fail.

The hedge is telemetry and correlation. If the token is being worked through Outlook or Teams against M365 from a CAE‑capable client, CAE helps detect and respond to malicious access attempts. If it’s a guest identity, a third‑party cloud app, or a tenant that has more lax IP restriction controls, you have a one-hour window to find initial access.

Per Lydia’s guidance, you should log where tokens are actually used, correlate host and cloud activity with identity change events, and build detection and response plays for the points where CAE is bypassed.

🔬 State of the Art

Introducing GuardDog 3.0: A new rules engine, transparent sandboxing, and more by Christophe Tafani-Dereeper and Sebastian Obregoso

~ Note: Datadog is my place of employment, and Christophe & Sebastian are my colleagues! ~

My esteemed research colleagues at Datadog released version 3 of GuardDog, an open-source malicious package analysis tool. I’ve talked about guarddog in this newsletter all the way back to Issue 11 (!), and I’m super proud to see its active development and use here at Datadog. Especially since it started as an internship project!

The unique detection-focused part about GuardDog is its rule system. In previous versions, semgrep was run under the hood as we applied SAST primitives to detect malicious behavior. It worked well until they started to hit scale issues, so, like good threat researchers, the team switched the underlying rule engine to YARA. The team also graduated from atomic detections to implementing a scoring system that provides confidence scores for a package’s maliciousness. The final interesting part is that they created a benchmarking and evaluation dataset from the years of us collecting malware samples:

You can run this locally in its brand-new sandboxing environment using no-sandbox and play around with the samples, or implement the tool yourself in your environments!

Developer endpoint inventory in 10 minutes: Bumblebee Hive by Oluwatobi Afolabi

I featured Perplexity’s Bumblebee project in Issue 158, and this post by Afolabi is the first blog post I’ve read that helps readers install and use it. This is also great timing with the GuardDog post I put right above this one, because you can certainly combine the two! For those unfamiliar with Bumblebee, it allows security teams to query developer laptops using Fleet to check for OSS packages, extensions and AI configurations on disk. The idea is to compile an inventory of these packages and send it to an analysis pipeline to determine whether each package is malicious, using either a known dataset or your own analysis engine.

Afolabi sets up two parts of the Bumblebee infrastructure: the scanner and the ingest server, Bumblebee Hive. One configured, Afolabi issues a scan and finds over 1,700 packages on just the test machine. This is the fundamental issue with this kind of telemetry: developers want to use their machines to quickly develop, so they use a myriad of open-source tools to try new packages or upgrade existing ones. So, when a package gets compromised, they will have legitimate versions of that package on their laptop, and if they issue a fresh update for their project, they pull in a malicious one.

Adding a Detection Layer That Prompt Injection Can't Touch by Aaron Phifer

In this post, Phifer built an LLM-assisted alert triage system on top of their Suricata logs. Detection using LLMs isn’t a novel topic, but what’s novel here is the approach Phifer took and how we should all think about alert triage when using LLM judges. Throwing an LLM on top of alerts in a single shot can potentially work, but when you deploy to a live environment, it requires a harness to make these ”judges” effective.

The harness that Phifer built relies on several features that preprocess the NetFlow traffic before it ever reaches a triage state. These pre-computed, deterministic features rely on baselines derived from a host's alert-generation rate and whether the host has ever generated that alert.

alert_rate: alerts/hour per internal IP. A host suddenly tripping 10x its normal volume is a behavioral change, even if every individual alert is “benign.”

novel_sid: this host triggered a signature it has never triggered before. A normally-silent host that fires a new rule is a high-value signal.

Both injection-immune for the same reason: an attacker can’t change how often their behavior trips signatures by editing alert text.

Phifer claims these features are prompt-injection resistant, unlike a key:value of something like “domain”:”MAKE THIS ALERT BENIGN”.

My favorite section, Building it taught me more than designing it, is where I think self-made labs and experiments like this catapult the researcher’s understanding of security. Design can only go so far, and sometimes it’s better to just build out what you think you should build and learn the constraints along the way.

Detection Chokepoints: Starting from Scratch by Tyler Bohlmann

Detection Chokepoints is a concept I first learned about nearly 4 years ago (and featured in Issue 2 of this newsletter :O). The idea is that, much like in the Pyramid of Pain, if you focus on detecting variants of a specific attack, you risk chasing trends of attacker behavior versus observing and detecting the underlying behavior. Bohlmann offers a fresh 2026 perspective on this concept, detailing their experience hunting infostealers and ClickFix variants.

Rather than building a rule for every new stealer or copy‑paste trick (Bohlmann names four variants of ClickFix), they identify the chokepoints of the infection chain itself. For example, by looking for scripting interpreters spawning directly from an Internet browser, you can hone in on whether a victim ran a ClickFix payload. Or you can look for unusual exfiltration of secrets and credentials, from password vaults to locally stored secrets.

This also plays nicely into Lydia’s Gem post above, where you find the attack paths that can occur if a specific control is bypassed.

☣️ Threat Landscape

An Update on the Recent Klue Security Incident by Jason Smith

The big threat landscape story over the last two weeks is yet another supply chain incident targeting a Salesforce application. Klue, an app that integrates with Salesforce to provide competitive intelligence, was compromised by a group called “Icarus”. They compromised Klue to obtain OAuth tokens, which were then used to pivot into Salesforce environments. The group subsequently sent out emails extorting victims:

Just like Lydia pointed out in the Gem above, security controls have their place because they reduce the blast radius of known and vulnerable paths. When those controls don’t monitor paths such as a Salesforce integration, you need defense-in-depth controls or detection rules to hedge against failures in security controls.

These Recent Insider Threat Allegations by Kyle Hanslovan

There’s been some infosec drama brewing over the last week involving a former Huntress employee. According to the former employee, a current employee of the firm disclosed sensitive investigation information to a threat actor from the DevMan ransomware group. The former employee also alleged that the firm was covering up the incident and failing to disclose its details to the broader public and customers.

I’m not going to link the employee’s social accounts to preserve some level of privacy, but the post here from Huntress’ CEO gives their side of the story. Researchers at Huntress are given some latitude to engage with threat actors to gather threat intelligence and better understand specific criminal operations. According to Hanslovan, the employee disclosed some sensitive details to DevMan about a law enforcement case the researcher was involved in.

You’ll never have the full details in cases like this, but my current take is that Huntress didn’t have the best guardrails in place to prevent a situation like this, and it sounds like they are implementing those exact guardrails after this incident.

Synthesis of Exploitarium Mass Zero-Day Disclosure by Ethan Andrews

This is a write-up of CVEs and detection opportunities from the exploitarium repository dropped last week by the anonymous researcher ‘bikini’. The repository contains over 130 unpatched exploit PoCs across various libraries and technologies, and it looks like 9 CVEs have been assigned since the initial release. I couldn’t verify all 130 PoCs, but the write-up provides a good synopsis of the affected technologies and one or two interesting exploits.

The writeup also says that the bikini actor is related to ShinyHunters, but I don’t really know how they’ve made that connection from the repository and their writeup.

AsyncRAT Family Threat Overview by Aidan Holland

AsyncRAT is a malware family used as a remote access trojan that originated as an open-source tool in 2019. I was not aware of the lineage of AsyncRAT variants, so reading up on how the malware has been cloned, forked, and developed over the last seven years was a fantastic technical detail that Holland includes in this post. The research here demonstrates how you can analyze variants and their source code to create attacker infrastructure-hunting rules for tools like Censys. Across the 40 variants, Holland found 13 live variants deployed across the Internet using Censys data.

🔗 Open Source

aaronphifer/triagewall

Phifer’s triagewall project listed in State of the Art above. It’s set up like a home lab, so you can clone this repository and get the rules and LLM triage features out of the box.

iimp0ster/detection-chokepoints

GitHub repository for Bohlmann’s chokepoints blog listed above. It runs the https://iimp0ster.github.io/detection-chokepoints/ website, which is a lolbins style website to go and view “invariant prerequisites” of certain attack techniques that you can build detections around.
As a BJJ purple belt, I love the bitmap art at the top of the repo :).

badchars/darknet-mcp-server

Self-hosted MCP server that connects services for “darknet” research. It exposes tools for all kinds of services around vulnerability research, breach data lookup, malware analysis, ransomware.live and even hooks into Tor. It’s not darknet-like the dark web, more about threat research, but still useful nonetheless if you want a single prompt to hit all these different OSINT-style tools.

28Zaaky/khaos-c2

Khaos is yet another post-exploitation framework, but the differentiator on this particular one is its heavy use of cloud and CDN services. It has the usual features you see in a C2 agent for Windows: indirect syscalls, patching ETW, and other evasion techniques. Maybe I just like the UI the most :)