Welcome to Issue #169 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• It’s getting a bit colder here in New England, but to me it's the perfect temperature. 70F/21C during the day and 55F/12C at night. Sweater weather and I can still wear some shorts :)

• My kids started school this week, and it's a huge transition for them. Shoutout to all the parents who deal with overtired and overstimulated kids as soon as they get off the bus, because ooo boy this is crazy!

• I’ll be in NYC this month and next month for work. I’m excited to be back, as it's been a few months, and it turns out I’ll be there during fashion week, so you can expect me to be dressed in couture. I just have to find some affordable couture first

💎 Detection Engineering Gem 💎

A scenario to evaluate your Agentic SOC by John Ao and Michel San

The commoditization of Agentic SOC is a good thing for all of us. And what I mean by this is that the techniques we see from evals, like the one in this blog, weren’t built in the open by AI SOC companies. This makes sense from a purely business perspective: evals and harnesses remain moats for startups, allowing them to show how differentiated their products are and sell to companies that need them. As general knowledge for agent architecture entered the security zeitgeist, these moats shrank and teams started to roll out their own capabilities. This is a natural evolution of any security capability.

Evals are so important for testing whether your harness will work in a live environment, and I’ve featured so many of these evals from blog posts, like from friends of the newsletter at Cotool. This blog by Ao and San feels like the most advanced because it covers a full end-to-end intrusion:

There’s a lot going on here, and I highly recommend clicking through to read their research, but it’s a TeamPCP-like compromise. It starts with a malicious pull-request workflow execution that poisons a cache inside a privileged GitHub Actions release build, where the poisoned image is run on Kubernetes, and the attacker priv esc’s out of the Kubernetes cluster compromise to gain full AWS account-level control.

They ran several frontier models through the scenario with Splunk API access. Five of seven assigned the initial alert a false-positive or benign-positive disposition, which is an immediate failure under their scorecard. Claude Opus 5 identified the attack as malicious, but achieved only 29% of the weighted investigation score, in part because it confused some malicious traffic with legitimate developer operations.

The validation gate is the most interesting here. The 29% number from above is a cumulation of scenarios each model correctly identified:

This report card raises the bar for evals in that it’s not just about precision or recall from True Positive/Benign/False Positive, but about the quality of the report. As it finds more and more parts of the attack chain, the model should provide evidence to the person investigating the intrusion. It also helps identify IOCs, distinguish legitimate activity from adversary behavior, and provide an analyst with enough evidence to scope and respond to the incident.

🔬 State of the Art

I’m in your logs now: deceiving analysts and blinding EDRs by Olaf Hartong

This blog by Hartong is a comprehensive deep dive into Windows telemetry and its use in security. Microsoft’s Event Tracing for Windows, or ETW, is a critical technology used by EDRs and SIEMs for performing detection & response. It has a few advantages: it’s a native technology shipped by Microsoft, so it’s a standard, supported way to collect telemetry; it has a pub-sub architecture to ensure you collect only the events you need; and it’s safer than shipping kernel modules that could brick a system.

Security tooling adopted it because it provides great visibility, but it was not designed with security in mind- more so observability. That distinction becomes important when security tools turn ETW events into cloud telemetry, detections, and analyst evidence, which is what Hartong’s research is about.

He starts with the useful defender goal of generating synthetic events to test detections without repeatedly emulating attacks as you would see on Atomic Red Team. He created his own ETW subscriber and producer, reproduced the data structures and payloads, and began sending them to Microsoft MDE. Once MDE began accepting telemetry, he began testing whether it could distinguish synthetic events sent from his tooling from actual events generated inside ETW.

Hartong essentially tricked MDE into believing the telemetry submitted on the local machine was authorized, without checking whether it was synthetic. Theoretically, this meant an attacker could submit confusing or incorrect telemetry from the Windows box on behalf of the ETW provider, which could trick a SOC analyst. I am unsure how likely this attack vector is, but IMHO, the point is more so that synthetic logs have their place in testing, but full end-to-end testing of an attack → alert helps guarantee that the full pipeline works.

A call for collective action on cyber defense by OpenAI

This security call-to-action letter from OpenAI is riding the tail of several disclosures that I’ve covered, including one from OpenAI itself, in which frontier models autonomously breached companies to solve a task given to them in a lab environment. It’s been making the rounds: 100+ companies signed it, including my own (Datadog), and I think the message is important, but I wonder if it’ll have any effect. Many open letters like this help grab headlines and signify commitment, but I don’t necessarily know what these companies and organizations will do differently after publication.

Now, I do think we need to stop being in awe of models that find vulnerabilities while stopping at the hard parts of patching and detection. I’d like to see more experiments and research into active agentic defense, where we build guardrails in a way that an agent stops an active attack without human intervention. This is the hard part because it’s based on trust rather than capability.

So who’s up for the challenge? :)

Inside Elastic's agentic SOC: How we took AI alert triage from 60% to 92% accuracy by Maggie Musquez

This post by Elastic’s internal SOC is similar to what I featured in Issue 166 from Figma. Elastic is a security company, and similar to my $DAYJOB, they are heavy dogfood users of their own products. They started rolling out an AI-SOC like triage agent and according to Musquez, it was hovering around 60% accuracy in terms of analysts agreeing with the agent’s analysis of an alert.

Much like Figma’s post and others in this space, the Elastic team began to build multiple agents to replace one-shot analysis. The goal here wasn’t just to ensure it reached the correct verdict: similar to what we saw in the Gem above, these investigative summaries needed to be actionable and of sufficient quality that a SOC analyst could quickly orient to the case and choose to escalate it or close it out.

They saw accuracy increases when they began adding context to the investigations themselves, primarily around:

• Past cases with the same rule name and making sure the investigation notebook inside the rule was high quality

• Workday enrichment to provide context around the employee itself

• Learning from past verdicts to steer the agents investigation

These enrichments, alongside more granular agents tasked with summarization, gathering evidence and pattern finding, moved their case accuracy to nearly 92%.

There’s some potential product pitches in the blog, but I’m not mad about it because the story was around dogfooding and building out in the open.

💬 Reddit SecOps Discussions

What is actually expected from a cybersecurity manager at a FAANG-level company?

It’s always funny reading Reddit discussions because there is a self-selection bias: those who respond tend to be negative. But I liked this thread because it gives a bit more insight into how management works, and doesn’t work, at a large tech company. I am also biased here in that “I” am a manager at a tech company, but for those who do want to get into leadership, especially in a SOC-like role, this is a great post to read.

Lazy Colleagues?

In this thread, the author, who works in a SOC, asked others about his colleagues who tend to pick up “easier” SOC tickets to complete, whereas he has more complicated ones. He ascribes this to laziness, and some replies suggest there may be a performance metric, such as the number of tickets closed or the time to close tickets. I think these are all fair points, but if I were mentoring this person, I would tell them to use their status as the “dependable one” to find more opportunities within the organization.

It’s a hard problem: the SOC is a grind, and you want to be a good teammate, but you likely have different aspirations than your colleagues because we are all unique in our goals. I would turn a situation like this into an opportunity for this person to start finding and proposing ideas to solve team or structural issues, as this is a clear indicator of a more senior analyst within a SOC.

☣️ Threat Landscape

Threat Road

Active PaperCut exploitation and the patch that changed overnight

What Happened…

Read more

7 days ago · Threat Road

I just found this publication on Substack, and what I appreciate about the way this post is structured is that it provides specific detection and learning advice for a vulnerability. PaperCut is a print management software used by companies to manage and deploy printers. I don’t necessarily know why someone would want this exposed to the Internet, but that was part of the attack path, not the two vulnerabilities released by PaperCut.

Threat Road highlights the full attack path for an attacker to gain privileged access to a PaperCut server within a network: the server must be Internet-facing, a configuration change must occur before an authorization check, and a separate deserialization step can turn the attack into an RCE.

They give practical detection opportunities in terms of what to look for and which logs you can use in your SIEM, as well as practice labs to understand the full attack path.

Improving our alignment and security efforts by Anthropic

Anthropic provided a post-mortem on their recent disclosures that early-release models broke out of the training environment sandbox and breached other environments. I do think some of this is interesting, and some of this feels.. standard practice?

They do take some time to explain how they are reconfiguring the model itself to make sure that it doesn’t perform harmful tasks when its given an impossible task (one that it can cheat on, such as CyberBench). I do find this bit sort of funny, though:

By default, all cyber evaluations should run inside a hardened sandbox (an isolated computing environment) with no internet access.

I wonder what they mean by "no Internet access," since OAI’s incident involved granting access to an Artifactory server to install packages. The models found an SSRF vulnerability inside that environment to achieve Internet access.

The Hugging Face Incident Is Not an AI Story by Marius Horatau

This blog post is a nearly perfect follow-on to the Anthropic post-mortem listed above. Horatau read OpenAI's Hugging Face Incident Report and shared his thoughts on some of the relatively basic isolation failures present in these eval environments. Although I enjoyed reading and watching the OpenAI post-mortem at Black Hat, the specifics of the breach were unremarkable from a detection standpoint. Horatau points out three failures:

• No security assessment on the sandbox environment. Horatau’s point is that this environment appears not to have gone through an assumed-breach assessment, or if it did, it was inadequate

• The alerts that did fire for the initial port scanning and message board triggered an incident, and the team contained it. There was no indication of any follow-up items to trace the attack path, and the agents began their exploitation runs again the next day

• It’s not apparent that any alerts fired on the security boundaries that seem standard to alert on. Any one alert on the sandbox gaining root on its host, a workload creating a privileged host-mounted pod, or a sandbox identity becoming Kubernetes cluster-admin should have fired an alert

🔗 Open Source

tailscale/tailcat

Tailcat is a netcat-like utility that you can use over a Tailscale connection for encrypted transfers. You don’t need the Tailscale control plane, but you still need an account to use it over their data plane. It does everything you’d expect a raw TCP listener/sender or netcat does: piping stdin/stdout, TCP forwarding, file transfer, SOCKS, and an exit-node mode.

DavidCarliez/cover

This is a neat sandbox-like project that performs inline replacement of secrets so your agents never see the “real” secret inside its environment. It’s essentially a proxy that replaces secrets with realistic stand-ins before they reach an external model, then it restores the originals only in responses that return to the local environment.

Azr43lKn1ght/DFIR-LABS

This repo hosts a free, community-built collection of Windows-focused digital forensics, incident response, malware-analysis, and threat-hunting challenges. Challenges range from easy to “insane” difficulty. It looks more like evidence-driven incident scenarios rather than only CTF flags.

Kerberosse/soc-dataset-thebiggerinterview

Ao and San’s investigation scenario used in their blog post from the Gem listed above. Sets up a local Splunk instance, ingests over 2 million events, and indexes them so you can point your own agents at it for evals.

Welcome to Issue #168 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I’m in my hometown this week with family, and I forget how much I love NY weather during the summer. You can throw a rock in any direction and likely hit a lake or a river here!

• For the first time in a long time, I neglected to bring running shoes and workout clothes to an extended vacation. I’m dying over here: I’m eating way too much food and have no counterbalance to work all of the calories off. I basically need to sneak away during the day and do pushups in a corner of my room if I want any semblance of a workout

• I keep getting notifications from Substack that my newsletter is part of a leaderboard for “Rising in Technology”. But everytime I check the board I don’t see my name :( maybe the leaderboard is so competitive I only get it for a few hours at a time

  

Sponsor: Detections.ai

Coverage analysis, tuned to your environment

detections.ai Enterprise builds a baseline of the tech you run, the detections you already have, and threats you care about. New intel gets analyzed against that baseline continuously and automatically, with notifications when we find gaps.

You get coverage answers specific to your environment. No generic MITRE ATT&CK percentages: what applies to you, what you already cover, what’s a gap, and why.

Detections tuned to your data sources and existing rules, ready to sync back to your stack.

Book A Walkthrough

💎 Detection Engineering Gem 💎

Where do Detection Ideas Come From? by Gary Katz and Jason Deyalsingh

I’ve written a ton about detection backlogs, even dating back to Issue 4 in 2022 (!). In my years of building a detection engineering function, the detection backlog and its management have been a hot topic in planning work for the quarter or even the next few weeks. You try to cover and document as much as you can, and we typically ask the questions:

• Do we have the right breadth of coverage?

  • Did we onboard any new technologies that we need to log?

  • Are we covering every tactic in ATT&CK for an attack surface, such as host or cloud?

• Do we have the right depth of coverage?

  • How many techniques are we covering in ATT&CK?

  • Were there any new red team exercises that discovered gaps in catching techniques?

Note that the questions are insular, as in, they don’t encapsulate what threat actors are doing targeting a business that looks similar to yours. I can spend so much time writing rules for Active Directory, but if Active Directory accounts for a small percentage of the environment, then it may not be the best use of time.

That is why I like Katz and Deyalsingh’s framing in this blog. They describe four places from which detection ideas can originate: visibility (breadth), existing detections (depth), the environment (context), and threat-intelligence research (environment X the threat landscape). The useful part here is not picking one as the right starting point. It is using the other three as filters.

They make the argument that a rule request or a Sigma rule you find online is not automatically backlog-worthy. They ask and answer much better questions than I did earlier. You should look at the existing visibility: is there existing coverage, or are the relevant adversaries using the technique against a company like yours? The same logic applies in reverse when you start with a new log source, a critical asset, or a threat report.

🔬 State of the Art

How Well Can Open-Weight Models Evade SIEM Detection? by Daniel Knight

With the latest news about frontier, low-guardrail models breaching legitimate companies, it’s good to see open research into how similar incidents could occur with an open-weight model. In this post, Knight emulates a small- to medium-sized business lab environment with a full Wazuh stack to show how their open-weight model, Sable, combined with other models, can successfully compromise the environment.

The testing showed that their autonomous agent could reason over an Active Directory environment, validate a multi-stage ACL delegation path, select lower-artifact techniques, and complete a domain-level compromise without generating a high-severity correlated incident in Wazuh. The goal here wasn’t to compare and contrast the model's expertise in this environment. What I took from this was that the lack of guardrails within open-weight models is the barrier to entry to this type of setup.

The other interesting things was that the model was smart enough to do things like sleep in between attacks so that it can outlast correlation windows, use as much living-off-the-land binaries as it could, and a custom in-process HTTPS C2 framework:

The beacon was designed as a lightweight, memory-resident HTTPS polling channel rather than a conventional installed implant. Contrary to popular opinion, you can’t expect to run Winpeas and not get caught. It ran within WS01’s existing WinRM worker process, wsmprovhost, under a Kerberos-authenticated administrative session. The agent used a dedicated HTTPS endpoint, certificate validation, encrypted task exchanges, randomized callback timing, error backoff, and a defined kill date. Commands were queued by the control service and their results returned during later callbacks.

I do have some good news, or at least hopeful news, after reading this post. I think we are going to see more experimentation with the efficacy of open-weight models for adversary simulation, which means we need a separate response on the threat-hunting and detection side of the house. This includes being autonomous enough to deploy rules, hunts, and containment actions.

Beyond Detection: Getting Real Value From Sigma and YARA Rules Across EDR, NDR, XDR, and SIEM Tims Tittus

This is a great introductory post for those trying to understand both the differences among detection products and the applicable rules. Tittus does a particularly good job of distinguishing between writing detection rules against logs using Sigma, which should catch behavior, and writing YARA rules against files and memory, which is strictly pattern matching. They both have their place in the detection stack with their corresponding strengths and pitfalls.

I copy-pasted the table into ray.so for some prettier formatting:

I’ve generally viewed YARA as the earliest or latest stage of an investigation. The early stage is identifying known malware patterns and making a block-or-alert decision in EDR or NDR. The EDR can generate a log with an enriched view of the binary and YARA output. It’s also a great threat-hunting tool, where you don’t mind sifting through false positives to help tune the rule and avoid overwhelming the SOC.

Self-Study in The Age of LLMs by Jared Stroud

At DEFCON, I caught Jared’s talk on finding vulnerabilities in bootloaders via directed fuzzing. It’s his PhD research for his PhD, and as someone very unfamiliar with fuzzing in general, it was a fascinating approach that combined static & semantic code analysis rules with LLMs to find entire vulnerability classes. After his talk, we were walking around the DEFCON floor, and I asked him for his thoughts on using LLMs in PhD research and what he likes and doesn’t like about them.

I felt somewhat validated because he wrote a blog post afterward about how he uses them as a self-study tool and “adversarial advisor” to help him understand complex topics, papers, and codebases. This is very similar to what I wrote in February earlier this year. Basically, if you can prompt chat and coding agents in ways that serve you as a learning or study buddy, it can significantly accelerate your understanding of a subject rather than trusting the output.

Jared warns about this in the post: it’s not about quizzing yourself and trying to recount answers in a cliff notes style quiz, but rather, using them to play roles or create labs for yourself to complete. For example, when trying to understand a codebase, Jared has the agent create a multi-module, graduate-level learning lab split between several chapters. He can work through each lab and break the work up in a way where it’s not overwhelming. I think the same can be applied to detection engineering and security operations concepts, rather than jumping right into a class of detection rules without understanding the underlying behavior of an attack.

💬 Reddit SecOps Discussions

Log everything, I’m begging you

I liked reading this discussion because it highlights the two biggest pain points for security teams: visibility and cost. There is no rule without telemetry, but if you aren’t being cognizant of cost while logging everything you can, you’ll be drowned out by a large logging bill or too many alerts that can overwhelm a security operations team.

There are certainly strategies to help reduce cost while still logging as much as you can. Many solutions decouple storage and compute, where you can store lower-value logs, such as DNS or HTTP traffic, and rehydrate them only during an investigation.

The key argument here is intentionality IMHO. If you just “log everything”, you’ll lose on the cost front and frustrate coworkers and your finance department. You may want to keep ALL DNS logs, and as long as you have a good business justification, it shouldn’t be an issue. Perhaps you do have great rules or a very specific compliance requirement to do so, but the key is to write that risk down and make sure people understand it.

Does a SOC have to constantly justify its existence?

Maybe I’m biased since I’ve been in the industry for 15 years, but I do think that it is much harder for people to break into security now than when I first started working. A lot of social media posts naturally highlight the negatives of security: costs, difficulty finding jobs, layoffs, and security breaches are all topics I see across different platforms. So when I see a post like this, I worry that, as a community, we tend to present the negatives to others rather than the positives. But isn’t that what the Internet is for?

Security teams are there as a hedge against control failures. So, no, a SOC doesn’t need to justify its existence, because I try to internalize that if a threat actor makes its way into my organization and detection rules are the last line of defense, there were many failures that happened along the way. I think this is the perspective that new grads should bring into their first SOC jobs. When you identify a threat actor, work quickly to contain the breach, and let yourself shine as you write the post-mortem. What controls failed? What classes of rules were missing? Can you remove rules after you implement a new control, because the rules become less relevant?

This approach is where I see entry-level security people become very successful. Security is a cost center, but that doesn’t mean we can’t help appropriate that cost in different ways.

☣️ Threat Landscape

More Criticals, Less Dopamine by Ananda Dhakal

I feel like vulnerability researchers are having an identity crisis over the last year. Large language models, starting with Opus, began challenging human expertise due to their speed in orienting to new codebases. A singular prompt and getting a cup of coffee can create an expert vulnerability researcher for a codebase. This wasn’t the case before, as Dhakal puts it, where the painstaking process of learning the codebase was part of the journey.

Although we’ve leapfrogged our capabilities for vulnerability analysis by throwing code at agents, we still haven’t clarified what it means to resolve and patch vulnerabilities. This is where I see vulnerability researchers getting stuck the most. The dopamine hit may not be as strong when we find a PoC with Opus, but IMHO the hard work is shifting towards patching and prevention. That work is severely underserved, and we could use help from brilliant researchers on solving that at scale. It’s not really impressive to me anymore, since people find vulnerabilities so easily; the barrier to entry for finding them is much lower.

I want to keep doing vulnerability research. But, I want to do something more than just abusing llms and hoping they pop a shell.

Post-DEF CON Phishing Uses Google Doc Apps Script to Deliver Malware by Jonathan Semon, Ryan Dowd and Ben Nahorney

Huntress Researchers uncovered a ClickFix social engineering campaign that was unique for two reasons. The first reason is that it specifically targeted security researchers on X during DEFCON. A fake X account reached out to researchers both publicly and via DMs, trying to get them to navigate to a malicious Google Doc. This is the really cool part, and forgive me for the screenshot because it won’t let me link to their video (though you should go watch it!)

The lure is a typical encrypted document pretext. The attacker sends a decryption key to the victim; the victim tries to paste it into a custom sidebar on the right, but it displays an error and prompts the victim to copy-paste the ClickFix payload.

The clever tactic is the custom sidebar itself. Apparently, in Google Docs, you can load a custom Google Docs App Script to automate certain tasks that help interact with the document itself. The Huntress team reverse-engineered the App Script and found a reusable template for malware delivery. It’s basically like a JavaScript payload but inside a trusted container. It sent victim information to a Telegram bot and served malware based on whether the user was on macOS or Windows.

We burned 11.7bn tokens to find the best cyber AI model by Philippe Dourassov and Debarshi

Aikido Research created a CVE benchmarking harness to measure the discovery of recently disclosed CVEs by leading Frontier models, both open and closed weights. They assessed 32 recently disclosed vulnerabilities, with each model receiving three independent attempts. They then combined each model's attempts, which materially improved recall, but it also created more candidate findings for the rest of the pipeline to validate.

Much like Daniel Knight’s blog post above, this is a reminder that the model is only one part of the system. The discovery harness, number of runs, cost, validation workflow, and tolerance for false positives are just as important as the precision and recall metrics on the final chart. I am getting more and more excited about these open-weight models because they’ll hopefully drive costs down and put more purchasing power in end users' hands.

🔗 Open Source

sunlife3/needre

neEDRe is a small, experimental Linux EDR built in Rust with Aya, an eBPF library. To me, the repository reads more like a learning environment and project than a full-blown EDR, but this is the kind of project that detection engineers need to do more of to understand how these technologies work under the hodo. Following Jared’s blog listed above in State of the Art, you can ask an agent to build a curriculum to understand how eBPF works under the hood and to extend the detection capabilities in neEDRe.

horsicq/XPEViewer

Cross-platform PE file viewer and editor that runs on Windows, Linux, and macOS. It is designed to inspect Portable Executable internals without requiring a full reversing environment, making it useful for fast malware triage and detection writing. There are plenty of tools in this category, but I thought this was interesting due to the cross platform support and if you need it in a pinch.

Zyrexnn/Cybermes

Cybermes is an autonomous offensive-security and bug-bounty framework built around the Hermes Agent, several Go utilities, MCP integrations, and more than 200 skills. It wraps security tools such as subfinder, httpx, katana, ffuf, nuclei, and sqlmap, writes the raw output to disk, then uses a custom tool called smart_pipe to extract the meaningful output to save on token and context window costs.

It’s gotten a ton of stars on GitHub recently, so I’d be interested to see if anyone is running this with any modicum of success. Hermes is the new OpenClaw and people seem to really like it, so a security version of it for bug bounty might be fun to run.

armourinfosec/Offensive-File-Transfer-Techniques

This repository is a collection of file-transfer techniques for Windows and Linux that can serve as a useful lexicon for those who want to build detection around exfiltration or transferring of data to evade detection.

It has a good split of modules and case studies to look through. For example, the section on netcat and Socat has some classic red team tradecraft on using these tried-and-true tools to get around egress filtering. IMHO, pick the transfer methods that map to your environment, run them in a lab, and identify which telemetry you can use to detect.

Welcome to Issue #167 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I had a fantastic time at Hacker Summer Camp! Thank you to everyone who came to my Happy Hour, where we had nearly 300 registrants!

• I did an awesome panel with and on SIEM and security operations hosted by Decibel Partners. Shoutout to for wrangling us together and giving me the creative space to try something a little different with the format

  

• Last but not least, I hosted a Detection Engineering panel at the Blue Team Village with leading experts in the space: Christina Parry, Chris Kulakowski, Alex Hurtado and Chase Phelps. According to the organizers of the Village, it was one of the largest turnouts they’ve ever had. The line to get into the talk stretched well beyond the village.

  

Sponsor:

Your security architecture matters more than your AI model

Security vendors are racing to ship better AI models, but their architectures are dragging down their models. Even peak AI can’t overcome a limitation like a global detection model shared by all customers.

Join Josh Kamdjou (CEO, Sublime) and Dmitri Alperovitch (Co-Founder, CrowdStrike) on September 10 to hear why detection architecture determines what defenders can actually do to close gaps, and find out what it looks like when your platform can autonomously close gaps in hours, not on a lengthy vendor update cadence.

Register Now

💎 Detection Engineering Gem 💎

Every time I attend Hacker Summer Camp, I try to survey the thousands of talks and build a rough logistical plan on how I can attend them. The plan never works because Vegas is way harder to navigate between the casinos, taxi lines and long Uber waits, but if I can attend 20% then I know I’ve done well. So when I saw this talk on the HuggingFace incident by OpenAI security on the schedule, I got excited because it was featured on my newsletter and it spawned subsequent disclosures by other labs.

To the surprise of no one, I was too far away at an event to attend this specific one, so luckily Black Hat published the video :). I highly recommend watching or at least listening to the briefing. The security incident details are, IMHO, insignificant in the sense that they are nothing unique to what we haven’t seen before. The details behind the evaluation environment is the most fascinating.

We know that the Foundational Labs have model evaluation environments to help determine the efficacy of their newest model releases. Of those evaluations, OpenAI and many others run ExploitGym as a cybersecurity benchmark. OpenAI’s ExploitGym environment did not have Internet access to make sure it couldn’t cheat, but it could use internal resources and tooling, such as Artifactory listed below.

The OpenAI team runs thousands of reinforcement learning evaluations on these models, so they give what they call “impossible tasks” to agents to see how they perform. These tasks can include writing an exploit for a vulnerability, or performing some calculation in a Google Sheet that requires Internet access. This helps guarantee that the models don’t “cheat” by using the Internet and can perform the task at hand. The impossible task runs across May yielded three distinct things:

• Agent runs found file write access (I’m unsure if this is a vulnerability, it was hard to tell in the presentation) to Artifactory which created a pseudo-message board scenario. Agents used this message board to communicate with each other

• An agent found an SSRF in Artifactory itself, which gave it internet access to help solve the impossible tasks

• A separate agent found an RCE in Artifactory from a legacy auth endpoint, giving it command and control access to Artifactory

These three things are important, because OAI found this activity prior to the Hugging Face Incident. They patched the vulnerabilities and redeployed the infrastructure. The issue was that this data was held in the memory of subsequent agent runs because the data made it back into normal agent evaluations. So they had a memory that this was possible!

The agents used this exact memory to re-create the message board and the subsequent attack paths through Artifactory into Hugging Face. They found different vulnerabilities and were so obsessed with their “impossible tasks” that it resulted in a real world breach. The agents determined that Hugging Face contained the shortest path answer to completing their evaluations, so they found a deserialization vulnerability in how Hugging Face loads dataset evaluations and chained that with a Jinja injection vulnerability to gain a foothold into Hugging Face’s environment.

I highly recommend watching this, it was a trip and one of my favorite talks of the conference.

🔬 State of the Art

Threat hunt AI: How we built an AI security analyst on AWS for under $500/month by Pablo Vidal

This post by Vidal outlines how their detection & response team runs their autonomous hunting and triage agent. It’s built on Serverless architecture on AWS: agents run as a container on AWS’ Elastic Container Service Fargate, and the agent loads in several hunting and detection configurations via YAML. It uses Snowflake as their longer term storage for alert and telemetry data while using 2 weeks of log, metric and traces data on Datadog (yay!). It integrates directly with their alerting and investigation workflow via JIRA and Slack.

The interesting design choice here is the multi-phase investigation. Sonnet is used as their data gathering agent, while Opus compares potentially malicious behavior to one- and seven-day baselines. It’ll enrich suspicious actors and resources, scores confidence, then uses a separate Opus agent to perform adversarial validation.

The final adversarial phase must produce specific counter-evidence before it can dismiss a finding, such as an allowlisted service account or an established false-positive pattern. Vidal says this reduced noise from roughly 40% of findings to a level where most results are worth analyst review. The title does say it ran for under $500/month, but that is an additive spend since we don’t know what their Datadog & Snowflake spend is.

Introducing Deputy: Better signal and control for software supply chains by Kent Gruber

Deputy is a rule-based language for detecting and enforcing security controls on vulnerabilities found in CI/CD. I included this blog in particular due to its similarities with how security operations deals with threats. The tool introduced in the blog, Deputy, helps find and alert on potential policy violations when code is being introduced into a source code pipeline. It uses a YAML based language, and the rule language is CEL, which we’ve featured on a previous issue with Perplexity’s numbat project.

There are nearly 60 policies in the repository, and each policy has several rules each that help enforce the policy. Enrichment is a key component to the policy engine, Deputy precomputes values to pass along to CEL so the rule language can alert on things like critical vulnerabilities, the presence of a vulnerability in KEV or a new container image that contains a critical vulnerability.

According to Gruber, the project is early and it will not replace reachability or application-context analysis. But the policy and rule model is worth studying. Supply-chain controls are more likely to survive when the same decision logic follows developers, CI, package proxies, and coding agents, and it introduces the funnel of fidelity effect on an attack surface not traditionally monitored by security operations engineers.

Detection Skills by Vega Threat Research

Detection-as-code has gained a ton of popularity in the last few years because it helped build resiliency in the quality of a rule as it moved through ideation, implementation, review and deployment. These pipelines help do everything from linting, to applying governance checks around reviewers and deploying only via Terraform, as well as applied testing via Atomic Red Team and the myriad of other testing suites. Agent skills are the next logical step of adding security agent steering-as-code to triage, response and detection engineering agents in prod.

I ran into Eli Rozen at the Vega booth at Black Hat, and he showed me their research around Detection Skills. This is a great concept: as detection engineers, we need to think about codifying the investigative and analytic knowledge we use for alert triage and tuning as much as the detection logic we deploy everyday.

The Detection Skills OpenSpec is Vega’s contribution to the detection engineering world that helps keep the detection logic in the detection platform, then attach a portable  SKILL.md  package containing the triage, investigation, and tuning work surrounding an alert.

A detection can start with a deterministic query, invoke an agentic triage skill, pass its result to an investigation skill, then send the outcome to a tuning skill that proposes a change for human review. According to this spec, skills can bring focused references, lookup tables, prebuilt queries, and enrichment scripts without loading every detail into an agent’s context window upfront.

The Vega Threat Research team published 50 skills with this initial release, so it’s really cool to see them build in the open and share their knowledge with everyone else.

ATEN: Endpoint Telemetry for AI Coding Agents by Anton Ovrutsky

Over the last two weeks, I’ve featured “AI-EDR”-like releases from Perplexity’s Numbat & Uber’s ADR projects. The development of security telemetry on coding agents makes a lot of sense. They have agent in the name, so much like an EDR agent on a host or a CNAPP agent running in the cloud, you can extend Claude-et-al to generate contextual coding session telemetry. This is the first post I’ve read on the concept of AI-EDR that brings receipts of how it works during an investigation.

In this post, Ovrutsky releases his ATEN project which provides coding agent security telemetry. Unlike Numbat & ADR, it’s observe only, but there are some unique elements to it that differ from those two projects:

• It reads each prompt and creates an intent trajectory, meaning it’ll enrich the prompt with additional information such as tool calls, agent session and results. This helps create an index of sessions to answer the who later on

• It uses Kernel-level collectors (Windows ETW, Linux eBPF) that records the action taken on behalf of the above intent. These collectors focus on host and network telemetry

• It combines the intent and the Kernel telemetry into a full picture, reconciling the underlying action of the prompt and tool call with the on-system activity

He has several examples of running prompts and viewing the output in Splunk. The Splunk queries and subsequent responses expose the intent and action from the session and gives a much richer view into what happened.

For example, he issued a prompt to run a health check inside a code repository, and the health check was a PS1 script that ran additional Claude prompts. The process chain is a mess if you look at EDR or ETW logs, but with ATEN, you can follow the chain-of-thought via the intent and the kernel actions in one view.

☣️ Threat Landscape

Lol agentic cyber breaches and the hype train

I am trying not to be facetious here, but I do want to point out some news announcements of organizations besides OAI/Anthropic who are disclosing security incident breaches due to training models with no cyber guardrails. I think it’s important we keep reading about these for two reasons:

1. It demonstrates that we need to seriously consider an agentic or autonomous approach to defense, as we will likely expect more of this in the future

2. The absolute craziness that these incidents come across more as cool than “dude you just breached my company”

Incident Report: unsanctioned agent behaviour during cyber testing by AI Security Institute

The AI Security Institute was running cyber evals with Mythos 5 and GPT-5.6-Sol with guardrails removed and found that less than 10% of their runs contained evidence of models trying to solve a challenge by attempting to breach organizations outside of their institute. The crazy finding here, IMHO, is that one of the models tried to insert malicious code into an open-source project and engaged in social engineering with forged identities to try to get maintainers to merge the project.

They released an in depth technical report alongside their announcement. The one thing I don’t necessarily understand here is that their eval setup permitted Internet access. OAI and Anthropic, for example, explicitly restrict Internet access, because they issue impossible tasks where they want to evaluate how the model reasons through solving the task without trying to cheat.

Meta says its AI model breached a third-party company during testing by CBS News

It was very difficult for me to find a source blog for this story, but I learned that it’s because Meta never released one. Meta spokespeople reached out to several news networks and disclosed that one of their models broke out of their training environment during training and breached an AI safety company called Irregular. Funny enough, Irregular has no blog or post-mortem on the breach, so I hope we can see one soon and see how the discovered vulnerability was exploited.

Other news

Zoomsday by Idan Levcovich

Levcovich released a blog on several vulnerabilities found by the A research team in Zoom. The research details how Zoom’s annotation protocol allowed an attacker to send a crafted annotation message that crossed a role boundary, reached a victim’s deserializer, and triggered memory-corruption bugs in the shared cross-platform annotation library. The deserialization part is interesting: according to Levcovich, Zoom has an undocumented serialization protocol with annotations to help it transcribe drawings on screen. They abused this serialization to achieve remote code execution.

I like that they included the prompts they used to help target Zoom. They decompiled the APK app for Android, and ran a threat modeling prompt, and discovered a shared library that contained the serialization protocol. Zoom issued 3 CVEs, 2 specifically for A Security and one for a separate vulnerability they found but was already disclosed to Zoom security.

ExfilSquad: New Data Extortion Group Leaks Microsoft D365 Data, Likely Linked to Misconfigured Power Pages by Fortra

TIL what Microsoft Power Pages are:

Quickly create enterprise-grade AI-driven business portals with built-in agents. 

According to Fortra researchers, ExfilSquad, a new data ransomware group, exploited default configurations in Power Pages to gain malicious access and exfiltrate data from 15 victim organizations. ExfilSquad released data from 13 organizations, totaling roughly 382 GB and 27 million records. The leaked JSONL and CRM/ERP fields are consistent with Dynamics 365 exports, and Fortra found no evidence of ransomware deployment.

In the rush of pushing out as many AI products as possible, Microsoft likely didn’t consider threat models surrounding products like Power Pages. Default configurations are tough: we learned how to harden things quickly when we ran our own servers internally after years of pain. The hard part for me here is that when I procure a SaaS product or deploy a product from a cloud provider like Microsoft, I would expect more attention to detail that we trust them with securely deploying it.

🔗 Open Source

CDESpace/Detection-Skills

GitHub link to Vega Threat Research’s detection skills blog. Contains the full specifications and example skills to inject into your own agentic workflows.

Antonlovesdnb/aten

Anton Ovrutsky’s repository to his ATEN project, featured in the State of the Art section above. The markdown and general documentation is amazing, especially the attribution section which ties an agent prompt turn to a kernel level action.

seifreed/CipherRun

CipherRun is a Rust-based TLS scanner that combines protocol and cipher-suite analysis, vulnerability checks, compliance controls, and large-scale scanning. It tests every version of TLS from SSLv2 all the way to TLS 1.3. Some other useful features and enrichment include checking for nearly 20 vulnerabilities, JA3/JA4 hash analysis and even monitoring certificate transparency logs for alerts.

AsaEdgerunner/linux-terminal-mastery

Open-source course focused on the Linux terminal that progresses from beginner command-line fundamentals to power-user workflows. Not security-specific, but a clean resource to hand to analysts or researchers who need to get more comfortable using Linux.

Welcome to Issue #166 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I’m on Day 2 of Hacker Summer Camp today! Yesterday was jam-packed with two great events. I helped lead a session on SIEM and Security Operations at Decibel’s Game Day, and we had a GREAT turnout at the Detection & Response Happy Hour. Almost 300 people signed up, and it was so great to meet people who read the newsletter and came to support!

• I’ll be walking the floor today and hitting up Olivia Gallucci’s talk on macOS Detection Engineering and an Agentic Detection session by two security engineers at Roblox

• My hotel lost my tshirt and stickers box. The FedEx in the hotel said the mail room has it, and the mail room swears that FedEx has it. So that’s gonna be a fun part of my morning :|
  

Sponsor: Material Security

Catching the phish was never the whole job.

Phishing's share of breaches hasn't moved in years, because that war’s been fought to a draw. Stolen sessions, dormant OAuth grants, and AI agents with more access than anyone tracks increasingly get attackers in instead. The perimeter held. The workspace didn't. See what detection looks like when it follows the attacker past the inbox.

See Workspace Security in Action

💎 Detection Engineering Gem 💎

How we secure Figma’s internal systems with agents by Matthew Sullivan and Brad Girardeau

Memory ended up being the thing that had the most impact on how useful the system became over time. We have several kinds, and keeping them separate turned out to be important.

What I love about reading modern security research in the blue team space is that as a community, we are applying more software engineering principles to security problems. This is why I love reading pieces like this, because it’s not just about agent design, but agent design for security operations AND it’s being used in the wild. The real contribution is not “agents for triage” in the abstract. It’s the claim that a security team’s value is in its expertise being recorded and reused instead of disappearing after an analyst makes a triage decision.

In this post, Sullivan and Girardeau reveal how Figma’s detection & response team built their own security alert triage orchestration system with a clever twist: encoding its efficacy and agent memories. Here’s how it works:

• For every alert triaged, they use a Retrieval Augmented Generation (RAG) to store analyst decisions, detection and alert metadata, investigative steps and memories for subsequent investigations

• When a subsequent alert is generated, their triage system queries the RAG database to search for similar alerts and uses that as context for the current investigation

• The agent posts a summary inside the Slack thread where the alert was used to page the on-call engineer, so when the engineer begins investigating, they have deeper context on how this was investigated before

• Once the engineer closes the alert, they update the Slack thread, which is then stored into the RAG system to create a closed loop

The team spent a lot of time describing their memory architecture and how it was the single best investment they’ve made to increase detection efficacy and reduce toil. Agent memory in this context is separated into three buckets: steering, procedural,, and case. Case memory is what you see in the picture above. Steering memory ensures an agent stays on track during an investigation, so it helps encode the agent's behavior during these steps. Lastly, procedural memory is the tribal knowledge that they store into agents as they navigate nuanced (see: janky) systems to make sure they know how to work with sharp edges.

I like to think about this type of setup as a “context pack”. LLMs thrive in context; it’s literally why they have a context window. Context engineering is a concept where you can inject instructions and knowledge into the context window to create consistency between agent runs. Figma’s three memory types, or context packs, ensure this consistency holds across the investigative, system, and behavioral steps of their alert triage.

The other thing to call out here is their confidence in letting agents create and tune detection rules. The memory each agent holds within each run helps steer the detection ruleset into a much more accurate state, but they do a good job of applying security boundaries so an agent is stopped right at the pull request, or when it tries to read Slack history or messages in channels that it isn’t explicitly allowed to read.

🔬 State of the Art

Securing Agents Across Perplexity’s Client Endpoints with Numbat by Perplexity

Coding agents are purposefully built to be non-deterministic. This non-determinism is what makes them fantastic at what they do. Personally, I have an agent for building and maintaining my home lab environment, a harness built for vulnerability research of malware and phishing panels and my agents at work that can orient themselves to codebases so I can get into a devloop for a service in a few minutes.

This non-determinism is what makes them dangerous, whether the intent was malicious or not. Last week, I linked to OpenAI & Hugging Face’s post-mortem on an unreleased OAI model breaching Hugging Face’s infrastructure. You’ll see in the threat landscape section below a similar announcement from Anthropic where it did nearly the same thing. I think the best way to secure your company against coding agents is not to assume whether they are malicious or not, but rather that they are untrusted.

Perplexity addressed this problem by releasing an open-source tool, dubbed Numbat, which is essentially an EDR plugin for coding agents. Coding agents expose rich functionality for hooking into their agents (Anthropic, OpenAI). These features are great for observability tracking but also double as a boon for security teams that try to track unsafe behavior for these agents. It integrates directly with over 10 coding agent harnesses and provides three security functions:

• Lifecycle hooks, which emit telemetry from events like tool use or filesystem usage on coding agents, which is useful for logging activity or blocking

• OTLP telemetry, which exposes an OpenTelemetry server that agents can submit logs and metrics to so it can be sent to a SIEM or logging platform

• Session artifacts, which help with forensics when an agent goes rogue, and you want to see how its context window led it astray (think of looking at raw prompt and response text)

The coolest part of this blog, though, is the rule language. Perplexity shipped an on-agent CEL integration, which is commonly used by Kubernetes operators to evaluate K8s control plane traffic, as their answer for logging and blocking. It reminds me of eBPF rules, where the agent emits an event, and the CEL connector reads incoming JSON and tries to match it against a rule to detect maliciousness. It has 52 (!) built-in rules located here, and it even supports correlated rule chains (check out this gnarly one for turning off agent guardrails, then the agent trying to egress data).

Latio Pulse

Detecting and Preventing the Hugging Face / OpenAI Incident

We link to some categories and companies in this blog, but this post isn’t sponsored in any way…

Read more

a month ago · 8 likes · James Berthoty

This week’s issue is certainly AI-security pilled. As we’ve seen from the Hugging Face/OpenAI & Anthropic disclosures, agent security is starting to decouple malicious intent from an agent’s execution. It’s not that intent matters; it’s more that our traditional models assume an attack inside your environments is likely the result of a malicious actor, and that doesn’t matter anymore. Attackers and agents both have goals; the difference is we allow agents to work inside our walled gardens.

My diatribe above is even harder to decompose when you have security companies shoving marketing down your throat. That’s why when I read this post from James, I was happy to see a practitioner’s approach to how the market can help against attacks like this. There are very few security market analysts I read and link in this newsletter, and James is at the top of my list.

James splits the Hugging Face/OpenAI attack into an ATT&CK-like chain, and helps readers understand what security product space could have helped detect or stop the attack along the way. He also presents these spaces as matter-of-fact: you don’t have to buy any of the products; instead, he frames it so that if you need to solve this issue, you can use those spaces as inspiration for your build-vs-buy decision. Honestly, this is the first analyst report I’ve read in a long time where I’d be happy to send this to my CSO :).

(by the way, if you haven’t subscribed to his Substack, please go do so!)

Deep-diving Crowdstrike’s DirectoryCreate by Daniel Koifman

EDR research posts like this tend to make my weekly reading because they help shed light on a very opaque product suite. In this post, Koifman aggregated CrowdStrike’s hundreds of logged event types and focused on a singular event: DirectoryCreate. What I love about this research is how focused it is. A seemingly tidy event tied to directory creation has a lot of nuance in how the telemetry is generated.

CrowdStrike’s sensor is developed as a minifilter, meaning it sits as a Kernel extension. In this scenario, when a user creates a directory, a user-mode function captures the request and calls the corresponding CreateDirectoryW. An internal manager creates an I/O request packet (a fancy way to say a data structure) and sends a structure called IRP_MJ_CREATE throughout the kernel extension stack until it hits CrowdStrike’s minifilter. This is different than hooking a function directly and it allows CrowdStrike to emit events for detection.

Koifman found several telemetry gaps using good-ol-Claude by creating 60 different ways to create a directory on Windows. I’m not a Windows expert by any means, but here’s what I learned from his findings that show how hard it is to reconcile telemetry from kernel to user-mode levels:

• Funny user-mode tricks to trick detections, such as specifying escaped or encoded paths, get resolved in the kernel. For example, if you pass a homoglyph into a directory string, the request packet normalizes the homoglyph to it’s true path in the filesystem. This means if you only look at the request packet for detection, you will likely not find obfuscation attempts

• There are many ways to create directories depending on how you interpret it. Things like renames, moves, recycle‑bin restores, and other lifecycle changes travel over a separate request packet  IRP_MJ_SET_INFORMATION and never surfaces as a  DirectoryCreate  event, so a folder staged in  C:\Temp  and moved into  C:\Sensitive  has no creation record at the final path.

• Reconciliation across events is just as important as precision on the singular event. CrowdStrike loses the attribution of who created the directory due to the IRP structure, so he recommends correlating that to a separate CrowdStrike event, which helps roll the process up to its original parent.

Sponsor: Rilevera

Make Your Detection Program More Resilient

Detection-as-Code is the core foundation for a resilient detection program.

Rilevera provides Detection-as-Code in the click of a button and gives you access to:

• Versioning & one-click rollback
• Peer review workflow
• Detection rule validation & efficacy reporting
• MITRE alignment mapping

Find us at Black Hat 2026 to learn more, or start a free trial now!

Get Started for Free

☣️ Threat Landscape

Investigating three real-world incidents in our cybersecurity evaluations by Anthropic

Following the Hugging Face/OpenAI breach, Anthropic’s security teams performed a retroactive review of their cyber evaluations to hunt for potentially similar malicious evaluations. After reviewing over 100,000 evaluation runs, they found three instances where their models gained unauthorized access to infrastructure that did not belong to Anthropic. I will say the three incidents weren’t as fancy as the Hugging Face breach, but it goes to show how these things don’t need to be fancy in order to be effective.

• In the first scenario, Claude discovered a real domain name and assumed it was part of the simulated environment, so it gained access via insecure application and database credentials

• The second scenario involved Claude publishing a malicious PyPI package in an attempt to gain access to a target company. This one was funnier to me because the model tried to obtain a phone number to register an account on PyPI and “..after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means”

• The last one did an Internet-wide scan, or at least used a service like Censys or Shodan, to find over 9,000 devices. Of those devices, it found insecure credentials again, and exploited the infrastructure to help complete its evaluation

If you refer back to my analysis of Perplexity’s numbat tool above, I talked about intent with these agents. At the end of the day, these incidents were the result of positive, misguided intent baked into the instructions of Anthropic agents. The intent didn’t matter since it still gained unauthorized access.

Amazon identifies North Korean hacker group behind open-source supply chain attacks by CJ Moses

Amazon Threat Intelligence published a campaign analysis of DPRK-linked threat actors abusing the open-source ecosystem, especially npm packages. There isn’t anything too interesting or surprising there, but I think the more interesting part is that they claim the tactics are shifting from singular compromises to multi-staged, multi-package compromises.

Their malware TTPs follow the same trajectory as cybercrime malware I’ve helped research for years and years. Two things that stand out for me is the use of licensing within the malware itself, and anti-sandboxing features. On the licensing TTP, they essentially install a reverse dead man’s switch: the malicious component executes in the package, but it will only run after it checks in with a DRM-style licensing server.

Anti-sandboxing is interesting because it’s an obvious next step for folks who know how malware tries to evade defenses, but it’s a bit harder to do with an interpretive language than machine code. I think it’s fine that it can attempt to detect whether or not it’s in an analysis environment, but the whole idea of these packages is that they run in a developer context, which looks like a sandbox environment. I’d be interested to see how this actually works or if someone has a sample, since Amazon Threat Intelligence did not share any hashes.

Popular npm Packages in the keyv and Cacheable Namespaces Compromised in Active Supply Chain Attack by Socket Research Team

A Shai-Hulud-style worm hit the keyv and cacheable npm packages, which have 150,000,000 and 10,000,000 weekly downloads, respectively. The initial access vector is unclear, but it does have a propagation element that steals discovered npm OIDC tokens and uses them to spread the malware. The malware has the same TTPs as most others in this space: it’ll steal cloud credentials and other keys and send them to attacker-controlled infrastructure.

The attacker-controlled infrastructure is the cool part here. Instead of a single hard-coded C2, the payload spins up fresh GitHub repositories via the API and uses commits and GraphQL  createCommitOnBranch  to stash stolen credentials, alongside a DNS-based exfil path for redundancy. It also plants autostart hooks in  .claude/settings.json  and  .vscode/tasks.json , so any developer or coding agent that opens the repo becomes infected.

The 73,000-server market reselling Western frontier AI into China by Lloyd Davies

I’ve read a lot about model distillation attacks, and there have been public accusations of the People’s Republic of China using unsanctioned access to Western frontier models to create their own open-weight versions. This is the first blog I’ve read that tracks the infrastructure that potentially makes this happen, and it reminds me of technique overlaps with residential proxy networks.

Infrawatch details the use of model transfer stations, where someone from a sanctioned country, such as China, can purchase proxy access to these models and pay a fee to use OpenAI/Claude and other Western models. These bridges consolidate their access to end users, and they use clever routing techniques from popular packages and coding harnesses like OpenRouter and LiteLLM to make it easy to use.

By operating this way, end users in China access the Great Firewall-approved infrastructure without touching blocked Western platforms, while those same platforms don’t block incoming connections from China because the proxy providers come from approved infrastructure.

🔗 Open Source

perplexityai/numbat

GitHub link to Perplexity’s “AI-EDR” project listed above in State of the Art. Check out the rules directory for some inspiration on detection opportunities for coding agent attacks.

uber/ADR

Uber open-sourced their own “AI-EDR” project dubbed ADR. The difference in this repository vs numbat is that they also released their own benchmarking datasets. This is great because it doesn’t only include tool call based detection and classification, but the conversations themselves.

Mihir4U-avi/GraphGulo

GraphGulo ingests pcap files and turns them into a locally run graph database for hunting and analysis. The cool part here is that it can run extremely fast queries and doesn’t run out of memory because it is backed by a Rust worker, making it easy to load in massive pcap files (the author boasts 10+ GB pcap files) with minimal impact on RAM.

supermhel/fengarde

Fengarde is a tool that takes disparate IT & OT logs from ICS telemetry sources, normalizes them over OCSF, and helps run Wazuh rules over them for threat detection. It normalizes and sends logs into OpenSearch, which Wazuh and several other SIEM providers support. It comes with 27 OOTB rules, and it’s cool to see what OT rules look like under the hood.

Welcome to Issue #165 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I will be at Hacker Summer Camp next week! I am hosting three events: two panels & a happy hour. I put the details at the end of the post in lieu of the Open Source Section. Come find me and grab a sticker!

• I spent time over the weekend at a lakehouse, and besides the pretty water views, there’s an eerie but beautiful sound I listen for at night. The Loon Wail is such a unique sound, and it’s something my kids get excited about every time they hear it.

• I’ve switched over nearly all of my coding agents to Pi. The customization and the ability to switch between model providers gives me Vim vibes.

detections.ai Enterprise is live. Private, coverage and context aware

A new threat drops and you need to move quickly. Run coverage analysis across your whole detection stack, build detections tuned directly for your environment, and deploy them back. In minutes, not days.

But coverage isn’t a one-time win. Detections drift, IOCs go stale, duplicate rules pile up. Our AI agents catch it while you sleep, so nothing slips through unnoticed, and your team spends its time on real threats, not upkeep.

Book a walkthrough

💎 Detection Engineering Gem 💎

Detections that run but can’t see by Nikhil Satyakrishna

SIEM and detection readiness is a topic I’m hearing more and more about in the research, security product, and internal operations spaces. I seem to link back to my Field Manual post a lot, but the line “There is No Rule without Telemetry” still holds true, but my post fails to address the underlying resilience gotchas. In this post, Satyakrishna addresses these issues and releases an open-source tool to demonstrate how to check for resilience failures.

The four failure modes worth keeping are:

• Disconnected: index pattern matches nothing

• Starved: sources exist but are stale or empty

• Missing fields: the field the rule filters on stopped being populated

• Ingest-lag blind window: events land after the rule’s window has already passed

If you asked me two years ago where the most important failure mode to check for is, it would be “starved”. A log source that doesn’t ship telemetry should be a red flag, but it’s much harder to check for than one would think. Is it not shipping telemetry because there is no telemetry to ship, or is there an underlying issue with the ingest pipeline?

Missing fields are a hard one to crack as well. You’ll need to understand the shape of the logs you are sending in, and make sure that if a log contains a missing field, it’s not due to a broken log source, an upstream provider changing the log envelope, or it’s expected behavior. As I’ve said before, we are becoming closer to SREs than we think :).

I’ve covered xcal’s work for the ingest-lag blind window issue using watermarks, but it’s nice to see another take on this on a log-agnostic use case:

Fraction of events a rule catches as source ingest lag grows, for a rule with a 6-minute lookback running every 5 minutes. The line is the formula, the open points are the simulation, and the shaded area is coverage lost with no error and no alert.

Satyakrishna released an Elastic-specific tool to check for the scenarios listed above plus two more, dubbed deadair. They also linked Elastic’s SIEM Readiness page, so it’s clear to me that research in the detection space is focusing more and more on detection readiness and resilience.

🔬 State of the Art

Agentic incident response notebooks by Kyrre Wahl Kongsgård

I’ve worked on hundreds of incidents at this point in my career, and it almost always falls into the same pattern. A Google Sheet gets shared, and by hour two everyone has 20+ tabs open: SIEM in one, threat intel platform in another, an asset inventory tool in a third, and the infamous scratchpad notes scattered in another. It makes sense, right? Security tools don’t do a great job of collating data, and giving yourself and your team more control over content curation means you need tools everyone can quickly orient to, to visualize the current state of the incident.

Kongsgård & the DNB’s Cyber Defense Center (authors from a previous Gem and building on work I featured almost 60 issues ago) built a super interesting agentic IR workflow on Marimo, where the notebook itself is the case file. Marimo beats Jupyter for this job for reasons that matter specifically to IR, such as Jupyter cells can run out of order, and an output can depend on state from a cell you already deleted, which isn’t helpful when you are trying to maintain chain of evidence or investigative steps.

Marimo gives responders a much more reactive experience. You change the host filter or the time window and every dependent query, join, and chart recomputes from the code actually on screen, versus pressing the play button on every subsequent Jupyter cell. Because the file is plain Python, the same notebook works as the analyst’s live workspace, so it’s great for source control.

They have several embedded examples that you can follow along, but my favorite part is their agentic loop for managing investigations via skills rather than building notebooks for each alert type:

The agent loop combines skills to manage the frontend investigation panel, specific domain skills on triaging and hunting through telemetry, and creating or maintaining sessions inside the investigative notebook. The domain skills help remove the complexity around dozens of notebooks per investigation because you can abstract the investigative patterns into markdown files versus code itself.

The Arguments Against Open Source AI are Very Bad by Tom Bedor

Last week’s issue contained stories around Kimi K3 and the Hugging Face/OpenAI model “breach”. It was brilliant timing for that issue: a Chinese company released a Frontier Labs-comparable open-weight model, and Hugging Face disclosed that they needed an open-weight model during their IR investigation since Frontier Labs APIs wouldn’t accept their investigative prompts due to cyber guardrails.

Bedor’s blog here is a reaction to the Kimi K3 blog and subsequent public reaction, where several prominent executives and leaders argue that open-weight models are too dangerous to release to the world. OpenAI’s Dean Ball specifically said:

One probable outcome of an open-weight-model-dominant world is full AI communism... rather than a market product, AI is a “public good”

Without open-weight models, could Hugging Face have reacted fast or efficiently enough? History is also not helping the argument for restricting open-weight access. Bedor cites several stories about the government trying to limit access to open-source software, such as SSL or PGP, and ultimately failing.

IMHO, his strongest point is that limiting who can inspect and run a model only serves attackers, since responsible actors are the ones who need unrestricted access to patch and investigate. HF’s own postmortem says the attacker was bound by no usage policy, while the defenders were the ones blocked by guardrails.

My advice for defenders: you should strongly consider open-weight models for investigation, especially if you don’t have access to lower guardrail models. It’s a discussion to have within your team all the way up to your CSO.

Accelerating EDR Evasion with LLM-Driven Analysis by Adam Chester

The Ralph Wiggum Loop is an agentic looping strategy where you shove a prompt into a simple infinite loop and have your coding agent work through the prompt indefinitely. It looks something like this:

The idea is that you assume your coding agent's prompts reach an eventual end, whether it’s through failure or through it completing its goal. It’s a harness that works so well that you see coding agents implement variations of the technique. This is especially useful when you can afford and want to run loops and trust that your agents can handle the goal prompt you have in mind, especially if it’s long-running analysis work.

This is where Chester’s blog falls nicely into place when you have goals surrounding cybersecurity analysis. In this post, Chester implements the Ralph Wiggum Loop to reverse-engineer Palo Alto’s Cortex EDR to find evasion opportunities. From a red teamer perspective, this is super helpful for several reasons:

• You farm out compute and mental load to a coding agent to help give you leads on evasion techniques

• The coding agent has access to the same tooling as you do, and can provide proof of its findings for you to verify

• The coding agent has context on previous runs, so that it doesn’t go down rabbit holes that it already went down, so it can focus on net new attack surfaces

The specific architecture revolves around maintaining three different markdown files:

REPORT.md – A markdown file used by the running agents to surface key findings for human review.

STATE.md – A state-file which each agent can use to track key events during analysis.

CODEMAP.md – Allows each agent to store references to areas of disassembly which are interesting or critical for analysis, helping increase velocity during later agent iterations.

AGENTS.md – A set of instructions to tell the model how to use the above files.

They found several bypasses in two separate buckets: extraction and evasion of loaded detection rules & ML models and evasion of the EDR functionality itself. The agent found, decrypted, and displayed thousands of YARA rules, which can be used in a separate harness to build evasions. It also found an evasion for Cortex’s user-mode DLL, which is used to enrich on-host telemetry and provide some level of hooking for prevention.

As far as vulnerability research is ahead of the game for agentic coding, I’d love to see more on the blue team side. A Ralph Wiggum Loop for threat hunting or threat detection seems like a good use of time and tokens!

Secondary Sponsor: Rilevera

Make Your Detection Program More Resilient

Detection-as-Code is the core foundation for a resilient detection program.

Rilevera provides Detection-as-Code in the click of a button and gives you access to:

• Versioning & one-click rollback
• Peer review workflow
• Detection rule validation & efficacy reporting
• MITRE alignment mapping

Find us at Black Hat 2026 to learn more, or start a free trial now!

Get Started for Free

☣️ Threat Landscape

LG to Ban Residential Proxies from Smart TV Apps by Brian Krebs

The TV manufacturer LG recently announced that it’s banning apps that fail to remove bundled residential proxy packages inside their app store. I’ve featured several stories related to residential proxies on this newsletter, including how botnet operators infect end-user devices and add victim nodes to botnets that get rented out to these proxy networks. This ban, according to Krebs, comes right after he posted a story citing Spur’s research leading to the NetNut botnet takedown.

What I Learned from Sitting in on a Scattered Spider Sentencing by John Fitzpatrick

This blog is the first time I’ve ever seen a security researcher sit in during a court hearing and sentencing to glean technical details on attacker tradecraft. Fitzpatrick sat in the sentencing hearing for two members of Scattered Spider who were responsible for the TfL breach.

The sad and maybe boring detail is around the initial access vector. Prosecutors walked through a compromised supply-chain vendor that built niche infrastructure software for TfL, from which the attackers pulled 857 rows of user records and credentials. Those credentials dated back to 2022 and sat in the criminal ecosystem for roughly two years before being used for the breach, which resulted in a months‑long outage and tens of millions in impact.

The probe that came 17 days before the CVE by HoneyLabs

As far as I can tell, there are two ways to find zerodays: research them yourself, or wait for an attacker to throw exploits at something you control and record that traffic. This is what HoneyLabs did with their global honeypot network, and this blog is especially interesting because it dives into their architecture around finding zerodays. They specifically recorded zero-day in cPanel scanning traffic 17 days before a CVE was issued, where 16 requests to one login path from a single IP, then silence, followed by mass scanning once the advisory dropped.

Their setup on building signatures is interesting. They only count a “pre‑CVE probe” if the signature is specific to a vulnerable code path, the reference date is the earliest public signal (NVD entry or an exploit template found in a project like Nuclei), and the traffic predates both their own sensor deployment and any public exploit chatter. They also throw away anything from noisy scanners or IPs that touch more than 50 different paths. After all that filtering, just 3 out of 1,697 CVEs in their corpus still show a genuine early probe.

Detection primitives for eBPF rootkits by Lorenzo Susini and Matt Muir

~ Note: Datadog is my current place of work, and Lorenzo & Matt are my colleagues! ~

My colleagues Lorenzo and Matt published research on common techniques used by modern eBPF rootkits. For those unfamiliar with eBPF, it’s a Linux kernel technology that lets small, sandboxed programs run inside the kernel to observe or influence system behavior. Its main use cases have been observability and networking, and it also has security applications for monitoring malware.

The same technology used for security can also be abused for malware. Lorenzo and Matt explore three malware families that use different techniques to evade detection. The challenge is that once a rootkit is loaded, the same eBPF instrumentation used to find security threats may be blind to its execution. At Datadog, we detect these programs by inspecting the load-time fingerprint, such as rare helper calls, program type, and other metadata, including attach type, name, tag, and map references.

🕵️‍♂️ Hacker Summer Camp

I’m so excited to be back at Hacker Summer Camp after taking a few years off! I have three events that I am hosting, so if you are around during these date and times, come hang out and say hello!

🍻 Detection & Response Happy Hour - Tuesday, August 4 5pm-8pm

I am hosting a pre-BlackHat Happy Hour with friends and co-sponsors Cotool and Nebulock. Food and drink will be provided, and I’ve been diligent on not letting sales people in ;)

Register Now

🪖 SIEM Under Siege - Who Will Survive? - Wednesday, August 5, 1pm-3pm

I will be at Decibel’s Game Day event hosting a panel with Jack Naglieri & Damien Lewke in an ESPN-style segment on SIEM. I have some great content already, and my hope is it won’t be a business-as-usual security discussion, and it will be way more spicy.

Come watch the show

The Modern Detection Engineer @ Blue Team Village - Friday, August 7, 345pm-445pm

I’m excited to announce that Detection Engineering Weekly is SPONSORING Blue Team Village this year! I’ll be there handing out stickers and talking shop with conference-goers, and on Day 1 of the Village, I’m hosting a panel with some excellent guests in the detection engineering space.

View the Panel Details

Welcome to Issue #164 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I took a 7-hour road trip back home while the smoke from the Canadian fires still filled the air. It was creepy and somewhat pretty in a “holy shit am I on Mars?” way, especially when I didn’t need my sunglasses in the middle of the day.

• The BlackHat Detection & Response Happy Hour hit capacity + waitlist in less than two weeks! Big shoutout to the co-sponsors, Cotool and Nebulock, for their support to help make this happen. I am going to try to bump up the capacity for the space even more, so register now if you can. I am also taking one more sponsor, so if you want to get your name in front of 100+ practitioners in the space, shoot me an email!

  Register for the Happy Hour

This Week’s Sponsor: Rilevera

Detections should be treated as code.

A rule misfiring in production is the most frustrating part of detection engineering. Attacker techniques shift, log sources change, and threat intel goes stale, you won’t know until it fails you mid-incident. Rilevera brings DevOps rigor to your detection program, from authoring through deployment.

Make your detection program more resilient with process-backed software and workflows. Rilevera gives you:

• Versioning & one-click rollback

• Peer review workflow

• Detection rule validation & efficacy reporting

• MITRE alignment mapping

Get Started for Free

💎 Detection Engineering Gem 💎

Your benign set should look malicious by Josh Waldrep

Detection & Response is the art and science of classification. I talk about labels a lot, especially when we start to consider rule efficacy as a function of a detection program’s success. It’s what SOC teams or incident responders immediately feel as pain. We may choose to incur the cost of a false positive for the sake of visibility, and in our heads we know it’s a good thing, but it’s still painful. This post by Waldrep helps explore labels while asking a critical question: “Are you being lazy with how you define false positives?”

The traditional AI or machine learning style labels include true positive (what we want), false positive (what we don’t want), true negative (what we want), and false negative (what we absolutely do not want). But, if you want to incur some cost for visibility, you may have heard the term “true positive benign”. It’s when you generate an alert where the traffic is benign, but you’ve accepted that you want to see it for a number of reasons, such as it being a critical asset or the rate is within acceptable parameters.

Waldrep’s argument, though, directly attacks the concept of a true positive benign and calls these types of rules lazy detectors. Using AI Agent detections as examples, you can see situations where a detection is lazy because the benign set you use to test isn’t using real data but rather synthetic data:

These are all fantastic examples of where, as Waldrep puts it, a detection did not have a realistic benign set for backtesting. It’s especially “lazy” because building benign sets is just as hard as building malicious sets because it’s rooted in your environmental context versus something you see online. For example, in the base64 image data URI scenario, you can alert on every base64 string you see. But if you aren’t feeding your backtesting set with images versus what is maliciously decoded base64, you may be testing on the wrong set of data altogether.

The ending quote is a pithy recap of everything in the blog post:

A detection earns its false-positive rate by staying calm when benign traffic looks like an attack. If it never met dangerous-looking benign traffic in testing, it hasn’t earned anything yet, no matter how good the number looks.

🔬 State of the Art

The Centralized Detection Model and Its Limits by Josh Kamdjou and Ian Thiel

I am happy to see this blog by Josh & Ian, who are friends of the newsletter, get published for folks to read! Full disclosure: I helped review the copy and theme for the Sublime team, but it is a great study of how email detection engineering is a different beast from pure SIEMs. This blog is Part 1 of 4, and it specifically addresses how email security vendors manage their detections under a “Centralized Detection Model”:

Most SIEM deployments you see combine vendor-provided rules, a.k.a. out-of-the-box rules, with custom or community rules. This makes sense in many ways: a threat actor who performs a malicious action within an environment, such as the cloud or an Active Directory domain, generates the same telemetry. You can replicate these attacks as closely as possible in your real environment and determine whether vendor or community rules reliably detect them.

The way I’ve interpreted this blog and series is that, as much as we think e-mail security is solved, which it looks like it isn’t, platforms that allow us to do modern threat detection have nascent capabilities to do so.

According to Josh and Ian, e-mail security vendors employ out-of-the-box rules just like SIEMs, but they tend to be more opaque. You can certainly replicate attacks over email, but if the underlying detection architecture doesn’t allow you to create, modify, or tune rules based on the environmental context of your organization, you risk becoming stuck while waiting for a vendor to update rules.

The team outlines three issues:

• Explainability of ML & AI models is hard, and providing rudimentary configuration tools to adjust a threshold (0-100 scores) isn’t helpful. The more models you add, the harder it is to understand evaluations of detections, so when the only thing you can control is that damn slider to move it up and down, it’s not helpful

• Suppressions and environmental context. Adding an IP address or a hostname to an allowlist is commonplace in detection. It’s brittle, but it does work on a per-rule or per-scenario basis. According to the team, many vendors provide a global allowlist, which can solve a false positive on one rule but create a false negative in the other

• Emerging threats and relying on vendors to create or update rules. They call this “mean-time-to-coverage”, but basically, when a novel attack or vulnerability drops, you need to quickly build coverage. When you have to rely on an upstream vendor to ship a rule, you lose agency in the ownership of your rules

I’d be interested to hear how e-mail security vendors publish rules and provide these capabilities for detection engineers. Sublime does offer its ruleset as open source, and it has a community version you can integrate into your personal infrastructure. I promise this isn’t a shill; I’ve known the Sublime team for years and love their detection-engineering-first approach to a field older than most of us.

Stop Using OpenCode by Luke Wren

Ok, this is a very irreverent post complaining about OpenCode, but I promise once you get towards the end you’ll start laughing. I’ve been messing with open source coding harnesses lately, and Pi has been pretty great to work with. It’s very configurable and not really opinionated until you start adding a ton of plugins and configurations to it. This is by design: for example, if I want it to use subagents, I have to install a subagent plugin.

It’s split up into two sections, Annoying Things & Alarming Things. Annoying Things seem to be more rooted in harness engineering decisions themselves, such as how it manages memory and its context windowing. The “Alarming Things” section is where the security stuff starts and where I started cracking up. The wildest part to me is how permissions are persisted across sessions, whereas in Claude Code you’ll have a fresh slate every time you close and re-open the session.

I won’t spoil too much, but this quote from Wren is exactly the vibe sprinkled throughout the piece:

Clankers are not (usually) malicious but they are naturally adversarial because they are trained to compensate for stupidity with persistence. This is not a guardrail, it’s thoughts and prayers.

Kimi K3: Open Frontier Intelligence by Moonshot AI

The big news over the last week is the release of an open-source model, Kimi K3, that, according to their benchmarks, performs as well or better than leading AI Frontier Lab models. News like this is always crazy timing when labs like Anthropic are releasing their best models ever under Fable or OpenAI’s GPT-5.6 Sol. The Frontier Models are called Frontier because they a) perform the best and b) are clutched closely by the companies that train them due to fear of alignment.

Basically, if Mythos were open-source, would you trust those who can clone it into their Mac Mini to do the right thing hacking-wise? These labs generally say no. But when you see a release of a model that performs really well against the leading models, it makes you question whether or not that question is relevant because we’ll eventually democratize access to insanely performant models whether you pay for it or not.

They have several benchmarks, but the key graph is the cost versus score per task benchmark:

The economies of scale will likely protect OpenAI & Claude since they have moats other than their models, but this makes a compelling case for people to reduce cost for nearly the same performance.

☣️ Threat Landscape

Security incident disclosure — July 2026 by Hugging Face

Hugging Face, a very popular GitHub-style hosting provider for AI models, disclosed a breach from earlier this month. They claimed the breach was run, start to finish, by an autonomous AI agent, with no human operator confirmed at any stage. The attacker abused a vulnerability in Hugging Face’s data processing pipeline, where a malicious model managed to break out onto their Kubernetes infrastructure. From there, they harvested credentials and laterally moved throughout their environment. They claim there was no evidence of tampering with their or customers’ models.

The interesting tidbit here, though, is that as they were performing their forensic investigation, they couldn’t use Frontier Models for analysis due to usage and guardrail limits. Their security team had 10s of thousands of logs and commands to parse, and they were subsequently blocked from analyzing them with these Frontier Labs. Cleverly, the Hugging Face team hosted a local model which helped with the guardrail issues, and it cost them a lot less.

There’s nothing here about how they came to the conclusion that it was fully autonomous via an agent other than this:

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness - used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This matches the “agentic attacker” scenario the industry has been forecasting.

OpenAI and Hugging Face partner to address security incident during model evaluation by OpenAI

OpenAI fills in the missing half of the Hugging Face story on what “autonomous agent framework” found and exploited the vulnerabilities outlined in Hugging Face’s security incident disclosure above. According to their post, the “autonomous agent framework” that popped Hugging Face was built by OpenAI running an internal evaluation harness.

They were testing GPT‑5.6 “Sol” plus a more powerful unreleased model on ExploitGym, a public security benchmark, with internet access and guardrails relaxed. The models realized the answers lived on Hugging Face, then chained vulnerabilities across OpenAI’s research environment and Hugging Face’s dataset‑processing pipeline to pull the solutions straight out of Hugging Face’s production database.

This is wild, right? It’s showing how determined models become when given a task or a goal they need to complete, and without the right guardrails in place, they can certainly complete that task with some crazy tradecraft. It certainly makes the case for the need of security alignment in light of the Kimi release news in the State of the Art section above.

🎙️ Podcasts

I’m back to listening to more podcasts, so I’ll post one or more episodes in a separate section. They will sometimes be new episodes, but I do think there is some great content that I’ll post that can be months to years old.

Pacific Rim is a threat group tracked by Sophos that specializes in finding exploits against routers, VPN and networking appliances. This specific episode follows several current and former Sophos employees who helped discover a Chinese-nexus APT that specifically targeted their Sophos Firewall product lines.

It’s a wild story because once they found out that the trial VM running the firewall was being used for developing exploits, they pushed an “implant” into APT’s firewall appliance to monitor them. They would download the threat actors' files, exploits, research, and even flood the network with ARP requests to fingerprint other victim appliances. It sounded to me like they were APTing the APT :).

🔗 Open Source

luckyPipewrench/agent-egress-bench

Pipelab’s eval benchmark for malicious agent egress attacks. It splits different attacks across 18 categories that you can use as a way to evaluate your internal detections. The key here is that this tests the tools catching these attacks from your agents, not the agents themselves.

manishrawat21/Cisa-KEV-Threat-Intel-Orchestrator

“Zero-touch” CISA KEV vulnerability-to-detection pipeline. It pipes vulnerabilities with known exploits into a structured prompt in Gemini to generate a Sigma rule. It tries to replicate the traffic with the correct Sysmon EventIDs and has several filtering mechanisms so it applies suppressions so the rule isn’t too noisy. Interestingly, it builds a log of discovered vulnerabilities to rule implemented as a compliance tracking mechanism inside a Google Sheet.

MatheuZSecurity/Furtex

Furtex is a fairly comprehensive post-exploitation toolkit for Linux. It uses several techniques across two persistence techniques: io_uring and eBPF. It’s 100+ tools across four directories, and the interesting part here is its anti-EDR features. It targets Falco’s default ruleset directly, mapping all 25 rules to specific bypass tools.

The first technique is to stop the event from reaching Falco via clever io_uring manipulation, and the second lets the event through while breaking the rule condition via process name spoofing, path pivots, or alternate syscall flags. It’s cool to see anti-EDR techniques on Linux!

optimuslabs-io/grokpatrol

Grokpatrol is an open-source tool that checks if your Grok client uploaded your git repos and corresponding secrets to Google Cloud Storage infrastructure controlled by xAi. There was an excellent wire-level analysis on a Gist here, but tl;dr a background collector in the Grok CLI performed the upload outside the permission system of Grok itself. Grokpatrol scans several Grok artifacts for the “malicious” build that resulted in the DevOops upload.

Welcome to Issue #163 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• The sky and sun look crazy here in the Northeast U.S. due to fires a bit further north in Ontario. I hope anyone in that area stays safe and hopefully we get some rain here soon. I was driving towards the city here and drove by two brush fires, something I’ve NEVER seen before growing up in New York

• I’ve been ramping up training for a 10k race here in Maine called the Beach to Beacon. I used to run races all of the time until my life was taken over by BJJ. Happy to say it’s been going really well and I’m feeling confident I can hit my 10k goal time of 50:00 min or less :D

• I finished my build-your-own landline phone project. FreePBX took the longest since there’s no official Docker image so the VM install takes an hour, but other than that, the configuration of the phone and the trunk to Telnyx went super smooth. My kids now have a phone they can call family with!

Detection & Response Happy Hour @ Black Hat - ALMOST AT CAPACITY :O

If you are going to be in Vegas during Black Hat, come swing by Tom’s Watch Bar @ the NYNY Casino right on the strip on Tuesday!

I’m running it back after BSides SF/RSA with friends and supporters of the newsletter, Cotool.ai. We are almost at our 80 person capacity after one week, so register now before I have to figure out how to pay for more food and drinks :P

I opened up one more co-sponsor slot, so if you are interested in supporting the Happy Hour and having your company logo on the invite and the shoutouts in the newsletter & on social media, reach out to me at techy@detectionengineering.net

Otherwise, hit the register button below and come steal my stickers:

Register Now!

💎 Detection Engineering Gem 💎

Detection Engineering in the Era of Semantic Malware by Daniel Koifman

Semantic malware, or “promptware”, is malware attached to unstructured files and configurations used by coding agents. This research by Origin, which Koifman here referenced, shows that traditional threat detection strategies and mechanisms need to adapt to malware being injected into a non-deterministic process, a.k.a. coding agents. I talked about this type of malware with Agent Skills in last week’s issue. Koifman’s blog here highlights gaps in understanding and telemetry as we hunt for detection opportunities in these chaotic processes.

The fundamentals remain the same: we’ve seen work on detecting maliciousness in the agent's execution layer (also from last week’s post). Koifman points this out with areas to focus on in the underlying operating system, using process trees as an example. An exploit that runs on a piece of software and then causes PowerShell to run is an observable you can write a rule against. We are seeing this with IDEs and developer tools when they install packages. Now, we need to look at how coding agents themselves perform acts under malicious conditions, because some of our assumptions break:

Process tree analysis fares no better. In conventional detection, parent-child relationships are signal-rich: winword.exe spawning cmd.exe is suspicious; svchost.exe spawning powershell.exe is actionable. But an AI agent runtime is designed to spawn arbitrary processes. Claude Code calling curl, git, python, node, grep, and bash in rapid succession is not an anomaly — it is Tuesday. The agent’s process tree is a superset of almost every LOLBin chain an attacker would want. When the legitimate behavior of a process includes “execute any command the user or the context window tells it to,” there is no parent-child relationship left that constitutes an anomaly by itself.

He sheds some hope in the section afterward, where least privilege and some boring security controls can actually make a big difference. For example, he recommends file integrity monitoring (FIM) on critical coding agent configuration files and generating an alert if a process monitors a CLAUDE.md or similar file that is outside an allowed group of processes. IMHO, this is harder to do in practice. We’ve seen supply chain attacks target IDE extensions, so if your FIM rule allows disk writes to a CLAUDE.md if it came from Cursor, then a malicious extension can easily bypass the detection.

I don’t think this is a pure recommendation, but rather a challenge for us to consider the behavior and assumptions surrounding coding agents so we can build guardrails for them. EDR still works once the coding agent touches the operating system. Application allow listing, when configured correctly, can prevent the installation of post-exploitation malware, so you can build around the chaos of an agent. But that’s the thing: they are chaotic by design, and if you have a business that expects your engineering team to build and ship faster with these agents, you need to consider what friction you introduce into the environment that can slow them down as part of your risk calculation.

🔬 State of the Art

CTI-REALM: A new benchmark for end-to-end detection rule generation with AI agents by Arjun Chakraborty

CTI-REALM is an open-source benchmark for detection, ideation, and implementation from CTI reports. Chakraborty and Microsoft Researchers curated 30+ threat reports from industry sources (and they named Datadog Security Labs, where I work, as one of them n.b.d) and used them as a source of truth. They then measured how foundational models ingested these reports, built and explored telemetry associated with these attacks, and generated rules on the other end.

Interestingly, they contributed this dataset directly to the UK government’s AI Inspect repository, which hosts datasets and methodologies for evaluating AI tools.

Why I Spend More Time Proving Myself Wrong Than Hunting Threats by Smruti Ranjan Pradhan

This was a great post that helps readers demystify the true value of threat hunting. I think we teach threat hunting as the practice of finding security incidents and threat actors that our detection/SOC team missed, and that’s the only thing threat hunters do. There is a big problem with this: 99% of the time, you won’t find anything. But that is by design because there is a lot more to hunting that contributes even more value besides doing hunts that “fail” over and over again.

Pradhan hits the nail on the head, describing hunting as a way to check biases in your detection strategy, discover gaps, and learn more about your company’s environment. This sets threat hunting teams up for success because, in my humble opinion (and how I organize our threat hunting effort here at Datadog), the goal of threat hunting is to discover gaps. Testing a hypothesis by searching your telemetry to disprove it should lead you to identify control and detection gaps. These gaps should inform your security strategy and help answer the “are we covered?” question that every single leader and executive asks.

I’m super excited to see that my friend Matt Johansen (of the VulnU Newsletter, go subscribe!) and Ed Lowlevel (of lowlevel.tv) have started a new security podcast. They review the most interesting security news every week, and you can tell how much time and thoughtfulness they put into the production and quality here. What I appreciate here is how technical AND accessible they are.

The first half was the most interesting for me, and it revolved around the arrest of a Com member, Peter Stokes, because they stepped away from the marketing b.s. you see around “Scattered Spider”. Specifically, they called out how threat intel and cybercriminal investigations actually work, where you rely on the “bad guys” making OPSEC and privacy mistakes as soon as they lose interest in the problem space. They managed to nail Stokes using a GDID attached to his Windows O.S., and Microsoft helped correlate that ID with their web activity tied to Steam.

tl;dr: don’t play Steam games on the same operating system as your operational box :).

Defending SaaS-based applications against ShinyHunters OAuth abuse by Microsoft Security Research

Microsoft Security Research dropped a blog on TTP updates from ShinyHunters intrusions they’ve observed since 2025. The move to a supply chain compromise, paired with pure exfiltration, shows how effective this strategy can be without worrying about the ransom component. It certainly simplifies operations: you target SaaS environments and avoid EDRs altogether; you exploit nascent SaaS trust boundaries; and you rely on the lack of visibility into these attack paths to persist.

They certainly pitch their products as a way to showcase how they would protect against these attacks, but if you step away from the product pitch and focus on the solutions themselves, it all comes down to gaining visibility into third-party tooling. Unused, new, or risky applications granted via OAuth consent should be part of your risk calculation, and limiting the blast radius by removing these apps can really save you headaches if you fall victim to consent phishing or one of your vendors suffers a breach.

Hunting malware and malicious MCPs in memory on Kubernetes with FleetDM + Osquery + YARA by Ben Bornholm

TIL osquery can run YARA scans! Ben, a fellow RIT grad, created this step-by-step labs post to show how you can enroll devices into an osquery fleet and run YARA rules to scan for malware. He set up two scenarios to demonstrate the YARA plugin's scanning and alerting functionality. The first involved a K8S cluster with a Damn Vulnerable Web App container that he infected with Sliver, while the second involved a malicious MCP server.

The query is gnarly but effective: you left join your container queries with a remote, authenticated YARA rule and display the result if it finds a match:

From my experience with osquery, the thing you always have to worry about is latency causing problems by slowing endpoints down. YARA was purpose-built to be fast, so I’d be interested to see more research in this area and to hear how folks have deployed it at scale.

☣️ Threat Landscape

Alleged Member of Criminal Cyber Hacking Group “Scattered Spider” Arrested in Finland and Extradited to the United States by U.S. Department of Justice

Peter Stokes, an alleged member of The Com, was arrested in Finland and brought to the U.S. after being tied to several ransomware attacks. They racked up several charges across computer intrusion and fraud. The criminal complaint is unsealed, so you can go read it, but one thing I haven’t seen much in these announcements is that, about halfway down the blog, they tell a story about how Stokes failed to execute a ransom.

He was apparently involved in an intrusion into a luxury jeweler, and the security team at the firm managed to evict the threat actors from its network and prevent a major breach. It’s the first time I’ve seen a story about an intrusion's failure. They claimed it still cost $2 million in losses, but perhaps that was included to show that even carrying out these breaches can still result in a large loss for a company.

Compromised AsyncAPI npm packages: inside a CI supply-chain attack by Christophe Tafani-Dereeper, Eslam Salem and Sebastian Obregoso

My coworkers released a detailed campaign analysis where multiple asyncapi  npm packages were compromised on July 14, with malicious versions published directly from the project’s own  GitHub Actions workflow. A single commit injected obfuscated JavaScript into source files, resulting in four poisoned packages being published to npm. These four packages have a combined total of 3 million downloads per week.

This team always comes in the clutch with their timeline graphics, so here’s a great breakdown of everything they’ve found:

SpectrePaste by Joshua Platt and Jason Reaves

Walmart Threat Intel team were studying OSINT reports around DeepLoad campaigns and found a separate, previously undocumented PowerShell delivery system they call SpectrePaste. The fileless malware panel, internally labeled “PasteFast Panel” by the actors, has some impressive features that help operators manage large numbers of infections checking into the server without crashing it.

According to Platt and Reaves, the tool has AI-assisted development written all over it. But it’s not just things like clear Claude comments: they found several text files using Microsoft/GitHub’s spec-kit instruction sets. I linked spec-kit before, but you essentially use it to act as a product manager for your coding agents, and it’ll write code while checking back in with you on things like hitting milestones, getting clarification on features, and accepting changes.

NSA revives 'Tailored Access Operations' name for elite hacking unit by Martin Matishak

The National Security Agency’s famous “TAO” office has returned, according to the NSA Director. The announcement came last week and helps reunite “operators” and “developers”. This is NSA/government speak for security experts who specialize in the red team space and their counterparts who build and deploy tooling for the red teamers. I was always confused about why these groups were split off in the first place. I have a funny story about this group and Edward Snowden if y’all ever catch me in person at a meetup :).

🔗 Open Source

krovix-1902/detectionforge-

Detection Forge is an agent harness that ingests CTI reports and outputs detection rules in Sigma. It has several preprocessing steps to extract TTPs and IOCs, which then gets mapped to MITRE, and it iterates on building Sigma rules while self-correcting any errors that come along the way. After Sigma, it’ll convert to rule formats covered by Sigma converter backends, and it’ll open a P/R for review. Perhaps this would be a great candidate to do evals on using CTI-REALM from Microsoft above!

penberg/chimera

Chimera is a Rust-based sandbox for running untrusted code, specifically around security agents, to help reduce the blast radius of a malicious agent or a piece of malware being executed on your box. There are comparable projects like nono that perform similar functionality, but it looks like this implements a pseudo-hypervisor via a binary translator and converts them between architectures on the host and the guest.

Corgea/Sighthound

Sighthound is a tree-sitter based static application security testing (SAST) tool similar to Semgrep or GitHub’s CodeQL. It has the features you would expect: rule writing, taint analysis and integration with GitHub Code Scanning. It looks like you need to purchase Corgea’s platform to get other features enabled such as the AI SAST components, but it’s good to see competitors in this space to give us all options.

UKGovernmentBEIS/inspect_evals

UK Government’s LLM evals repository where the Microsoft Security Research team contributed their CTI-REALM benchmark. There’s over 10 cybersecurity benchmarks and datasets ranging from capture the flag harnesses, vulnerability research and incident response.

Welcome to Issue #162 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I had an excellent long weekend celebrating the 4th of July here in the U.S.! It was a good lead-up to a rather disappointing World Cup loss on Monday :(

• We recently bought a kids’ WiFi landline phone thing so they can call family and chit-chat. Let me tell you: it’s been terrible. The quality/service is poor, and it just feels cheap. So, I’m trying my hand at rolling my own FreePBX server with an upstream trunk provider. I haven’t been this excited about a project in a long time :D will report back once I get it deployed

Detection & Response Happy Hour @ Black Hat

If you are going to be in Vegas during Black Hat, come swing by Tom’s Watch Bar @ the NYNY Casino right on the strip on Tuesday!

I’m running it back after BSides/SFRSA with friends and supporters of the newsletter, Cotool.ai. It was super chill at RSA with no vendor b.s., so escape Mandalay Bay and come talk shop with other practitioners.

Register Now!

💎 Detection Engineering Gem 💎

End-to-end detection validation using coding agents by Kyrre Wahl Kongsgård

This blog is one of the single best deep dives on detection validation I’ve seen in years. It hits a bunch of themes, including types of detection testing and architectural decisions for building and deploying end-to-end tests, and clearly describes an elegant and repeatable agentic loop for this use case. Let’s break it down piece by piece, because there’s a ton here and I highly recommend reading this one if you don’t want to touch any other story in this issue.

Security folks tend to steal concepts from SRE and developers and relabel them with fancier, cooler names, but the underlying principles remain the same. The picture above from Kongsgård shows how we celebrate concepts like testing and “chaos engineering” in security and map them to the security telemetry lifecycle. Regression tests, for example, focus only on verifying that an input (telemetry) produces an output (alert). Synthetic ingestion is an integration test of the ingestion and shaping of telemetry to generate an alert. End-to-end testing looks at the full telemetry → detection → response pipeline.

My favorite themes

Since this post has enough content to fill several posts, I’m going to point out two of my favorite themes so it doesn’t feel like I’m repeating or rewriting Kongsgård’s content.

Lab environment and TTP framework

Booting labs up to run simulations takes a ton of time and effort, especially if you are starting from scratch. The goal is to replicate your environment as closely as possible, but there are always trade-offs in simulation. Some of these tradeoffs include:

• Environment mirroring: A host running in a VPC can help mirror what your endpoints or cloud resources look like, but it won’t be exact. Detonating potentially dangerous tooling inside a production environment can introduce externalities or even real security incidents if you aren’t careful

• Baseline activity: A user, much like an agent, is non-deterministic. The telemetry they generate from normal activity is just as important to model as the malicious traffic itself

• Provisioning discipline: Running a small amount of Atomic or Stratus Red Team tests is manageable from an individual detection engineer’s perspective. If you want to run your whole catalog of detections, you need to start thinking like an SRE or software engineer, as you’ll hit scaling and drift issues with your infrastructure

Kongsgård’s detonation environment has a high level of discipline to address these tradeoffs. The section on the Lab environment uses several DevOps paradigms, such as golden images, configuration management, and deployments via GitHub Action runners. Under the hood, they use Meta’s TTPForge as their adversary simulation framework for execution on detonation hosts. It offers a content-rich, multi-step attack-generation feature set that is adaptable to their agent-harness framework.

Agents as validation drivers and the schema knowledge base

Singular prompt, one-shot agents have their place in implementing agentic systems in security, but they tend to perform poorly as tasks become more heterogeneous. Since the task is end-to-end detection validation with TTP generation, rule tuning, and detonation, a Claude Code or Codex agent would not be sufficient.

The harness is the differentiator for anything agentic security. Remember that! Their agent isn’t improvising an attack; it’s following a plugin that teaches it to write TTPForge YAML files, ship them over SSH to a lab host, run the detonations, and then queries Splunk to see whether telemetry arrived and the detection matched. When a step fails, lifecycle hooks block progress until the agent finds the issue. Here’s the high-level architecture:

My recommendation to all my readers is to design your agentic workflows around single agents that do one thing very well. It’s as if you are extracting one piece of expertise from your brain and encoding it into a prompt to do a single thing. In this particular case, Kongsgård designed two plugins to perform discrete tasks.

• detectionkit builds the TTP definition via TTPForge, writes the detonation test, deploys and runs the detonation. Its singular purpose is to replicate threat actor activity in a common & repeatable lexicon

• splunk is the plugin that performs the validation that the correct telemetry was captured, the search for the rule was performed quickly, and continuously discovers index structure to understand rule performance and drift

Security vendors who sell agentic capabilities typically don’t expose their harnesses at the level of detail shown in this blog. Researchers and open-source enthusiasts are quickly catching up with these vendor-led harnesses, and this truly gives detection teams agency to choose between build and buy.

It’s been easier to admit that I can’t imagine a world without a Claude Code or Codex. The prompt was the star of the show for the first year or so of this coding agent frenzy, but it’s now squarely the quality of the harness that brings detection to the next level.

🔬 State of the Art

Detecting Agentic Threats in Claude: Writing Rules on the Execution Layer by Andrew Byford

This is a Part 2 post from Byford’s previously featured work on writing rules for Anthropic’s Compliance API. The cool part here is that, unlike his last post, which focused on the prompt content itself, this looks at the execution layer of the coding agents. I’ve always interpreted the execution layer as how the agent interacts with the filesystem itself. This presents unique detection challenges because, in my opinion, the impact is the same, such as downloading and executing a binary, but the paths are different, such as malicious skills, reading a malicious prompt, or loading a malicious plugin.

Byford splits the threat categories into five distinct buckets: excessive agency and permissions, supply chain threats, dangerous actions, sensitive information disclosure, and data poisoning. The architecture is clever where the Compliance API is used as an enrichment backdrop during investigations, so you can combine unstructured data from prompts with the structured data generated from Claude hooks:

And here’s the enrichment layer after a SIEM rule fires from the OTel collector:

Much like detection engineers have had to become supply chain security experts in the last two years, I don’t see a world where we also must become AI Coding agent experts in the next year or so. I never considered using prompt and response content generated from coding agents in the Compliance API as additional context for SIEM alerts, so I’m now going to steal that idea and see what I can do at my day job (sorry, Andrew!).

SOC Bench by DeepTempo

Evaluation datasets are critical for understanding model performance. Much like in my analysis of this week’s Gem, one-shot prompts can perform well under very constrained conditions, but without something to measure real-world malicious vs. real-world benign, you should limit your confidence in virtually all agentic security applications.

I found this SOCBench website & corresponding open-source repository, and it reminded me of Cotool’s Research benchmarks with similar datasets. This specific one includes a NetFlow dataset containing both malicious and benign network traffic. It’s also a bit more opinionated about persona benchmarks, ranging from SOC analyst to detection engineer, and includes more architecture, with tool catalogs and playbooks for those personas. Their first benchmark around detecting maliciousness:

Anthropic performed the best but it looks like it cost the most. I find it interesting that OpenAI’s benchmark had the threat analyst perform the best vs the SOC analyst in the other two.

Skills Registry Threat Models by Andrew Nesbitt

Two issues ago, I linked a blog by Aman Khurana that helped demystify the peculiar supply chain architecture behind VSCode extensions. The big takeaway I took from that blog is that not surprisingly, the more security engineers dig into supply chain security, the more they realize how difficult it is to piece together OSS ecosystems to perform effective detection and blocking. Coding agents are built to be autonomous and extensible, just like OSS. The difference lies in the non-deterministic way these agents perform coding tasks, due to intentionally designed boundaries.

In this post, Nesbitt unveils his threat model around coding agent skills. A skill is a bundle of prompts, code, dependencies, and tool permissions. Anytime a skill is used, the skill prompt is injected into the context window, and a set of tools and scripts gets exposed to the coding agent. The more frightening part of the Skills supply chain security is that, instead of a single npm command installing other packages in Node that eventually land a piece of malware, you can have a Skill install packages from virtually any ecosystem, and sometimes those packages are just more prompts.

I don’t think we are truly ready for a large-scale malicious Skill campaign, much like what we’ve seen with the likes of TeamPCP. Nesbitt points out several issues with how Skills are installed, deployed, and managed, and it certainly seems that this ecosystem is in the same stage that npm was in several years ago.

☣️ Threat Landscape

FBI Seizes NetNut Proxy Platform, Popa Botnet by Brian Krebs

The DoJ nabbed another residential proxy platform linked to the Popa Botnet. Krebs post here helps aggregate some of the data published by researchers at Google, Lumen and Spur. The wild part to me is that this proxy platform is linked to an Israeli company, and I’ve always assumed that these networks are owned by non-Western firms who are harder to work with outside of the U.S.’ sphere of influence.

These types of botnets finally figured out how to monetize without DDoSing. Krebs referenced research from Spur that nearly 50% of TV Apps on the LG Smart TV platform add the TV to these botnets, which are then sold as residential proxies.

I found a malware hiding in my tailwindcss config file. by Couch Potato

Super interesting write-up from a developer who encountered a Contagious Interview-style backdoor in their Tailwind configuration. They never figured out how it got there, but the indicators are classic Contagious Interview:

• Targeting developers and backdooring their code

• C2 server communication to an immutable blockchain style API

• Rewriting git history to conceal the compromise

It didn’t necessarily say what the impact was or whether the campaign resulted in data exfiltration. They did find several unknown processes running in their production environment, so likely something happened there. If I had to guess, it was a PwnRequest due to the rewriting of the git history, but that’s about as far as I’ll go before I start placing bets.

Linux Backdoor Targeting iKuai Routers by dmpdump

This is a cool Linux backdoor writeup of a piece of malware that, based on my ~limited research, targets a Chinese-focused router typically deployed to East Asian/Chinese businesses. It’s an ELF binary that impersonates OpenWRT’s libjson_script.so.0. It was hard to ascertain at first, but it certainly is not a shared library and runs in userland. I don’t necessarily know whether the victimology is Chinese firms, which could make this a Western-based piece of malware, but it seems compact and very specifically designed for one router brand, which makes it smell like an APT implant.

ARToken: Inside an EvilTokens affiliate panel targeting Microsoft 365 by Michael Kelley

The TALOS research team uncovered an offshoot of EvilTokens, a device-code phishing-as-a-service kit. Kelley uncovers the initial BEC-style lure and then reverse-engineers the kit to find modern front-end components, such as single-page application lures, and a full backend dashboard written in React. Some of the differentiating features of ARToken Kelley found include keyword searching across victim mailboxes, post-exploitation tooling against victim SharePoint servers, and even collaborative session links for operators working on the same ARToken deployment server.

Lydia Graslie’s Gem from last week helps protect against some of these attacks, especially if you monitor which Microsoft management surfaces emit audit data for device‑code flows and token lifecycles, and treat gaps and schema shifts as first‑class detection problems.

🔗 Open Source

DeepTempo/socbench

SOCBench’s open-source harness for evaluating alert datasets. The current dataset only contains NetFlow telemetry, but it looks like they want to add more. Their harness is the most interesting between playbooks, personas and how they run the evals themselves.

secdev02/EasyTokens

EasyTokens is a device code phishing toolset that emulates device code phishing as a service kit like EvilTokens. This one is more focused on performing the device code phishing attack itself, so you can use this to pivot into cloud and M365 environments.

kernelstub/Nox

Nox is an open-source attack surface scanning tool. There are 300 module plugins across 24 different categories. You can run each module individually or run a full scan that steps across all 24 categories to find everything from exposed credentials, vulnerabilities and OSINT findings.

phishdestroy/shortdot-evidence

Shortdot is a registry operator that hosts seven top-level zones (TLDs) that, according to PhishDestroy, almost exclusively contain fraud, phishing, and malware websites. This repository holds all of their research, enumerating the zones and their phishing website analysis across the seven sketchy-looking TLDs. Their research also includes financial research and how Shortdot charges ICANN fees to operate these zones.

Welcome to Issue #161 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I had an excellent vacation at the beach with my family! We stayed at an Airbnb with a 1-minute walk to the ocean. There’s something about the crashing waves and the smell of salty water that makes me wish I could afford a house there D:

• I am locked in & going to BSides LV/BlackHat/DEFCON! I’ll be posting details for my Detection & Response Happy Hour next week with sign-ups. Mark your calendars for Tuesday, Aug 4 at 5 pm :)

• I opened sponsorship slots up for the summer, so if you’d like to work with me on ad placements or opportunities to work with the Detection Engineering Weekly brand, shoot me an email: techy@detectionengineering.net

• Lastly, I opened a content submission page for folks who want to get their research and blogs in my reading queue. It’s much easier for me to use this then accidentally miss something on social media, Slack or e-mail!

  Submit a blog

💎 Detection Engineering Gem 💎

Control Plane

Where Continuous Access Evaluation Stops Being Continuous

Not every identity attack ends in a stolen token, but increasingly many do. Device-code phishing, adversary-in-the-middle proxies, OAuth consent grants: the front ends look nothing alike, yet they tend to converge on the same prize, a legitimate access or refresh token that Entra issued after a sign-in it had no reason to question. Sometimes that sign-i…

Read more

3 months ago · Lydia Graslie

Defense-in-depth is an overused phrase in security marketing, but it’s one of the few “buzzwords” where the definition matches in marketing-speak with what it means in security operations. At its core, detection & response is a hedge against when security controls fail. Examples of this include someone entering their username, password, and security code on a phishing page, or someone downloading an infostealer binary from an allow-listed domain, such as a CDN, and running it. The important part here is that detection engineers identify the attack paths that threat actors take when those controls fail.

Lydia’s blog (hi Lydia!) is a great example of the nuances of a powerful security control, Entra’s Continuous Access Evaluation (CAE), and how even the perfect implementation of that control can fail. Both infostealers and attacker-in-the-middle phishing pages are regularly stealing access tokens from victims, and when these tokens get into the hands of threat actors, they can use them to pivot into a production Entra environment. Microsoft implemented CAE to help combat long-lived tokens through a challenge/response mechanism to catch stolen tokens:

The idea is that legitimate or malicious access tokens should be evaluated against access policies and controls, and Entra can catch a stolen access token before a threat actor interacts with the target environment. It’s an excellent security control that is now the default for Entra environments, but much like multi-factor authentication, it has its sharp edges:

• There’s a 1-hour expiration window when the issuing client does not have a CAE-enabled auth flow

• Resources that don’t have CAE can still be interacted with, meaning a bypass of a CAE-enabled client against a non-CAE-enabled resource is possible

• IP restrictions can revoke the key quickly, but infostealers and phishing kits help provide geolocation and IP information, which can help bypass this restriction

Lydia provides a helpful coverage map for when each control fails and what you can do to “hedge” against a stolen token. This is where telemetry on hosts and cloud resources, combined with identity telemetry, provides a much stronger defense-in-depth approach when the best security controls fail.

The hedge is telemetry and correlation. If the token is being worked through Outlook or Teams against M365 from a CAE‑capable client, CAE helps detect and respond to malicious access attempts. If it’s a guest identity, a third‑party cloud app, or a tenant that has more lax IP restriction controls, you have a one-hour window to find initial access.

Per Lydia’s guidance, you should log where tokens are actually used, correlate host and cloud activity with identity change events, and build detection and response plays for the points where CAE is bypassed.

🔬 State of the Art

Introducing GuardDog 3.0: A new rules engine, transparent sandboxing, and more by Christophe Tafani-Dereeper and Sebastian Obregoso

~ Note: Datadog is my place of employment, and Christophe & Sebastian are my colleagues! ~

My esteemed research colleagues at Datadog released version 3 of GuardDog, an open-source malicious package analysis tool. I’ve talked about guarddog in this newsletter all the way back to Issue 11 (!), and I’m super proud to see its active development and use here at Datadog. Especially since it started as an internship project!

The unique detection-focused part about GuardDog is its rule system. In previous versions, semgrep was run under the hood as we applied SAST primitives to detect malicious behavior. It worked well until they started to hit scale issues, so, like good threat researchers, the team switched the underlying rule engine to YARA. The team also graduated from atomic detections to implementing a scoring system that provides confidence scores for a package’s maliciousness. The final interesting part is that they created a benchmarking and evaluation dataset from the years of us collecting malware samples:

You can run this locally in its brand-new sandboxing environment using no-sandbox and play around with the samples, or implement the tool yourself in your environments!

Developer endpoint inventory in 10 minutes: Bumblebee Hive by Oluwatobi Afolabi

I featured Perplexity’s Bumblebee project in Issue 158, and this post by Afolabi is the first blog post I’ve read that helps readers install and use it. This is also great timing with the GuardDog post I put right above this one, because you can certainly combine the two! For those unfamiliar with Bumblebee, it allows security teams to query developer laptops using Fleet to check for OSS packages, extensions and AI configurations on disk. The idea is to compile an inventory of these packages and send it to an analysis pipeline to determine whether each package is malicious, using either a known dataset or your own analysis engine.

Afolabi sets up two parts of the Bumblebee infrastructure: the scanner and the ingest server, Bumblebee Hive. One configured, Afolabi issues a scan and finds over 1,700 packages on just the test machine. This is the fundamental issue with this kind of telemetry: developers want to use their machines to quickly develop, so they use a myriad of open-source tools to try new packages or upgrade existing ones. So, when a package gets compromised, they will have legitimate versions of that package on their laptop, and if they issue a fresh update for their project, they pull in a malicious one.

Adding a Detection Layer That Prompt Injection Can't Touch by Aaron Phifer

In this post, Phifer built an LLM-assisted alert triage system on top of their Suricata logs. Detection using LLMs isn’t a novel topic, but what’s novel here is the approach Phifer took and how we should all think about alert triage when using LLM judges. Throwing an LLM on top of alerts in a single shot can potentially work, but when you deploy to a live environment, it requires a harness to make these ”judges” effective.

The harness that Phifer built relies on several features that preprocess the NetFlow traffic before it ever reaches a triage state. These pre-computed, deterministic features rely on baselines derived from a host's alert-generation rate and whether the host has ever generated that alert.

alert_rate: alerts/hour per internal IP. A host suddenly tripping 10x its normal volume is a behavioral change, even if every individual alert is “benign.”

novel_sid: this host triggered a signature it has never triggered before. A normally-silent host that fires a new rule is a high-value signal.

Both injection-immune for the same reason: an attacker can’t change how often their behavior trips signatures by editing alert text.

Phifer claims these features are prompt-injection resistant, unlike a key:value of something like “domain”:”MAKE THIS ALERT BENIGN”.

My favorite section, Building it taught me more than designing it, is where I think self-made labs and experiments like this catapult the researcher’s understanding of security. Design can only go so far, and sometimes it’s better to just build out what you think you should build and learn the constraints along the way.

Detection Chokepoints: Starting from Scratch by Tyler Bohlmann

Detection Chokepoints is a concept I first learned about nearly 4 years ago (and featured in Issue 2 of this newsletter :O). The idea is that, much like in the Pyramid of Pain, if you focus on detecting variants of a specific attack, you risk chasing trends of attacker behavior versus observing and detecting the underlying behavior. Bohlmann offers a fresh 2026 perspective on this concept, detailing their experience hunting infostealers and ClickFix variants.

Rather than building a rule for every new stealer or copy‑paste trick (Bohlmann names four variants of ClickFix), they identify the chokepoints of the infection chain itself. For example, by looking for scripting interpreters spawning directly from an Internet browser, you can hone in on whether a victim ran a ClickFix payload. Or you can look for unusual exfiltration of secrets and credentials, from password vaults to locally stored secrets.

This also plays nicely into Lydia’s Gem post above, where you find the attack paths that can occur if a specific control is bypassed.

☣️ Threat Landscape

An Update on the Recent Klue Security Incident by Jason Smith

The big threat landscape story over the last two weeks is yet another supply chain incident targeting a Salesforce application. Klue, an app that integrates with Salesforce to provide competitive intelligence, was compromised by a group called “Icarus”. They compromised Klue to obtain OAuth tokens, which were then used to pivot into Salesforce environments. The group subsequently sent out emails extorting victims:

Just like Lydia pointed out in the Gem above, security controls have their place because they reduce the blast radius of known and vulnerable paths. When those controls don’t monitor paths such as a Salesforce integration, you need defense-in-depth controls or detection rules to hedge against failures in security controls.

These Recent Insider Threat Allegations by Kyle Hanslovan

There’s been some infosec drama brewing over the last week involving a former Huntress employee. According to the former employee, a current employee of the firm disclosed sensitive investigation information to a threat actor from the DevMan ransomware group. The former employee also alleged that the firm was covering up the incident and failing to disclose its details to the broader public and customers.

I’m not going to link the employee’s social accounts to preserve some level of privacy, but the post here from Huntress’ CEO gives their side of the story. Researchers at Huntress are given some latitude to engage with threat actors to gather threat intelligence and better understand specific criminal operations. According to Hanslovan, the employee disclosed some sensitive details to DevMan about a law enforcement case the researcher was involved in.

You’ll never have the full details in cases like this, but my current take is that Huntress didn’t have the best guardrails in place to prevent a situation like this, and it sounds like they are implementing those exact guardrails after this incident.

Synthesis of Exploitarium Mass Zero-Day Disclosure by Ethan Andrews

This is a write-up of CVEs and detection opportunities from the exploitarium repository dropped last week by the anonymous researcher ‘bikini’. The repository contains over 130 unpatched exploit PoCs across various libraries and technologies, and it looks like 9 CVEs have been assigned since the initial release. I couldn’t verify all 130 PoCs, but the write-up provides a good synopsis of the affected technologies and one or two interesting exploits.

The writeup also says that the bikini actor is related to ShinyHunters, but I don’t really know how they’ve made that connection from the repository and their writeup.

AsyncRAT Family Threat Overview by Aidan Holland

AsyncRAT is a malware family used as a remote access trojan that originated as an open-source tool in 2019. I was not aware of the lineage of AsyncRAT variants, so reading up on how the malware has been cloned, forked, and developed over the last seven years was a fantastic technical detail that Holland includes in this post. The research here demonstrates how you can analyze variants and their source code to create attacker infrastructure-hunting rules for tools like Censys. Across the 40 variants, Holland found 13 live variants deployed across the Internet using Censys data.

🔗 Open Source

aaronphifer/triagewall

Phifer’s triagewall project listed in State of the Art above. It’s set up like a home lab, so you can clone this repository and get the rules and LLM triage features out of the box.

iimp0ster/detection-chokepoints

GitHub repository for Bohlmann’s chokepoints blog listed above. It runs the https://iimp0ster.github.io/detection-chokepoints/ website, which is a lolbins style website to go and view “invariant prerequisites” of certain attack techniques that you can build detections around.
As a BJJ purple belt, I love the bitmap art at the top of the repo :).

badchars/darknet-mcp-server

Self-hosted MCP server that connects services for “darknet” research. It exposes tools for all kinds of services around vulnerability research, breach data lookup, malware analysis, ransomware.live and even hooks into Tor. It’s not darknet-like the dark web, more about threat research, but still useful nonetheless if you want a single prompt to hit all these different OSINT-style tools.

28Zaaky/khaos-c2

Khaos is yet another post-exploitation framework, but the differentiator on this particular one is its heavy use of cloud and CDN services. It has the usual features you see in a C2 agent for Windows: indirect syscalls, patching ETW, and other evasion techniques. Maybe I just like the UI the most :)

Welcome to Issue #160 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I swear to God, maybe it’s because I’m in my 30s, but I am noticing so much more plant and flower life everywhere! I saw a huge bunch of flowers on the side of my yard, and when I asked Seek to classify them, they turned out to be blackberry bushes! Now I have blueberries and blackberries to pick when they start to produce :D

• I am taking the next week off for much-needed PTO, so the next DEW issue will go out on July 1!

• I am gauging interest for a Detection & Response Happy Hour during Black Hat week. Something lowkey, on the strip, where folks in D&R can meet without being barraged by vendors. If you are interested, hit the “Yes” option on the poll below so I can figure out seat count. I’m aiming for Tuesday night before the craziness starts!

  Every week, I read, watch and listen to all the Detection Engineering content so you can consume it all in 10 minutes. Subscribe and get a weekly digest of the latest and greatest in threat detection engineering!

💎 Detection Engineering Gem 💎

Detection Engineering’s Quant Era by Gabriel Abdelgawad

For those unfamiliar with the term Quant, it’s short for Quantitative Analyst. These analysts apply rigorous statistical methodologies to financial trading and sit at the intersection of business, mathematics, and market uncertainty. I’ve met several throughout my career, especially during my MBA. They enable massive trading firms on Wall Street to manage large trading portfolios that account for all kinds of risk. And, as we know about risk in security, they try to understand the uncertainty of events in the world, such as the Iran War, to find arbitrage opportunities or hedge against risks to minimize loss when a disaster happens. Sound familiar?

In this post, Abdelgawad surveys the history of quantitative analysis as it evolved from paper to computer spreadsheets, and now high-frequency trading. He compares and contrasts this evolution with that of security operations, especially given current AI capabilities. According to Abdelgaward, the cost of both writing results and triage is being reduced from a human capital perspective. A well-harnessed LLM can perform research, learn your detection stack, and build rules at a faster velocity than a human can. That same agentic system can triage alerts, analyze the event, and present a D&R engineer with its decision and reasoning.

So the question becomes: what do we do when the expensive parts are no longer typing rules or grinding through every alert by hand? Abdelgawad’s answer is that we are not being replaced, but we are being pushed up a layer. The work starts to look less like hand‑building each detection and more like portfolio management: deciding which rules belong in the book, which are brittle, which really work in your environment, and where the blind spots in the overall method are. He compares and contrasts this “squeeze” to an analogy of the front, middle and back offices in finance.

Once the front office (authoring) and the back office (false-positive minimization and risk acceptance) become cheap, the middle office becomes the constraint you optimize to achieve success in detection and response. So your “book” is your ruleset, detection pipeline, incident response playbooks, and your knowledge bases. To draw another comparison, the ship has sailed for writing most of your code by hand, as Claude Code has taken over. The same logic applies to detection: if agents can draft rules and triage alerts, the valuable human work is not clinging to manual authoring; it is managing the portfolio and the risk it represents.

I want to keep reiterating that the cost has shifted, and I’m hopeful because it frees up more time for detection engineers to focus on the important research and engineering parts of our job. Here are some ways I’ve seen this implemented already at my day job, where I run an org with dozens of security engineers doing detection & response:

• Teams move more into the threat hunting space, where they spend more time discovering gaps in coverage, telemetry, and infrastructure

• Projects emerge that help monitor drift in pipelines and telemetry and will flag when a rule or log source becomes unreliable

• Detection & response engineers spend more time with threat intelligence teams to help understand the world outside of the company

This has been one of my favorite posts to read in months. I highly recommend taking time to read and comprehend Abdelgawad’s narrative around this topic because we are going to be expected to take on more with the help of agentic tooling.

🔬 State of the Art

Statement on the US government directive to suspend access to Fable 5 and Mythos 5 by Anthropic

About a week after Anthropic’s release of the “Mythos-class” model Fable and Mythos 5, they pulled access to the model from all customers as well as non-U.S. citizens inside Anthropic. This blog post explains why they pulled access: the U.S. Government issued an export control directive requiring Anthropic to remove access for non-U.S. citizens.

A lot of work goes into releasing these models, including extensive internal and external red teaming to battle-test the jailbreak-prevention defenses that Foundational Labs add to them. According to Anthropic, they followed their normal operating testing procedures and added even more to make sure that the guardrails held up against adversarial prompts. They announced on their blog the intent behind the testing, since Mythos has been touted as an advanced, cybersecurity-capable model.

The blog reads as if they disagree with the directive altogether. According to the post, the government’s explanation was a singular report of a jailbreak as their evidence. This isn’t Anthropic’s first clash with the current Administration, in which the Secretary of Defense threatened to label them a supply chain risk.

My personal opinion is that this is likely a stretch of an argument by the U.S. and that I think this ban will be lifted in the coming months. That being said, Anthropic has leaned into the marketing of their Mythos model as being something that should be regulated, so I hope they aren’t too surprised that something like this happened.

Detecting Misuse with the Claude Compliance API: The Threat Is in the Content by Andrew Byford

I’m excited to see investment from detection engineers and researchers investigating how audit logs and compliance APIs work for AI tools like Claude & OpenAI. The problem with this particular threat surface, as Byford writes in this blog, is separating the threat model into SaaS detections and prompt-and-response detections. These APIs contain audit events similar to what we see across the industry: permissions or API keys being added to an account, MFA devices registered or removed and suspicious logins. But, if the value of these tools is token usage, how do you find threats in unstructured data?

Byford’s solution to this, which he also open-sourced, is a pipeline that splits control plane events under “Activity Feed” and content events. SIEMs, which are built on structured matching on structured data, perform poorly when matching on unstructured, non-deterministic data. These situations are when the problem (unstructured token usage) and the solution (unstructured token usage) are identical. Byford’s content pipeline uses a combination of prefiltering tools and an LLM judge to filter out interesting or malicious prompts before they ever hit a SIEM.

I do see some scale issues with this, but only in the sense that we will all likely face tradeoffs on volume vs precision:

• The cost of indexing prompts is shifted from the SIEM to the LLM judge. For every prompt your company generates, you need to use another prompt to evaluate its content

• Byford calls out the Judge being susceptible to attacks itself, so it’s important to engineer additional guardrails

• Privacy concerns around inspecting content in the prompt and uploaded documents themselves. Depending on your jurisdiction and compliance requirements, you may not get 100% visibility on every prompt and response. (Unless you are American, lol privacy)

Detecting and removing dangerous secrets on dev workstations before Shai-Hulud does by Guillaume Ross

This blog by Ross presents a practical pattern for secrets on dev workstations before infostealers or open-source supply chain attacks from groups like TeamPCP. They vibe-coded a proof-of-concept architecture that combines a secrets scanning tool, bagel, with Fleet and osquery. You can deploy the scan to run periodically via a LaunchAgent on macOS that runs bagel, which creates a findings JSON file.

Fleet then launches an osquery rule to detect secrets in the user’s home directory, sends alerts to Slack, and even instruments a response action to isolate the user’s access to your IdP. There’s a good defense-in-depth story here: you enforce controls before a developer gets infected, and you hope your EDR catches the infection before it’s too late.

☣️ Threat Landscape

Ransomware Tool Matrix Project Updates: Three Groups To Track by Will ‘BushidoToken’ Thomas

Threat research G.O.A.T. BushidoToken released an update to his ransomware tool and vulnerability matrix dataset. He added three emerging groups: TheGentlemen, Dragonforce and WarLock. It’s always difficult for me to stay abreast of TTPs in these groups since they tend to move fast, disappear and re-emerge. The coolest part of this post, IMHO, is the callout on the leaked chats from TheGentlemen and how they helped researchers understand the group's inner workings.

Phishing for Lobsters: How We Tricked OpenClaw into Spilling Secrets by Itay Yashar

In this post, Varonis Security Researcher Itay Yashar set up a simulated enterprise environment on Google Workspace and gave access to an OpenClaw agent. The clever idea here was to test the difference between an agent inadvertently executing malicious instructions via prompt injection versus what Yashar calls “agent phishing.” I think this is an important callout because it challenges the guardrails of a legitimate request with no hidden artifacts from a malicious source. Whereas in prompt injection, they are generally malicious requests with a hidden artifact.

They ran four scenarios, and the results were interesting: they tended to favor resolving issues over verifying the sender's identity and security. They implemented a “strict” verification profile for the agent that explicitly required it to verify identities. In multiple scenarios, a fake email was enough to get cloud credentials, secrets, and CRM exports. It just goes to show that guardrails are more than just prompts, and you should treat prompts and instructions as a security boundary in themselves.

Caught a ClickFix attack today. The domain name alone made me do a double take. (Reddit post) by MoneySaxena

I’ve been checking out Reddit posts more lately for raw analysis and commentary on detection and the threat landscape, and this one on ClickFix struck a good balance between technical depth and a conversational tone. Microsoft Defender fired an alert about a potential ClickFix compromise, and MoneySaxena wrote about their experience triaging the alert and containing the host.

They called the user and tried to understand how they managed to visit a site, copy and paste a malicious command, and then execute it. She said she was “just browsing normally”. This is something I see so many security teams deal with in their day-to-day, so it’s not too surprising that the user was browsing in a benign way and got infected.

This was my favorite quote from their write-up:

The thing that gets me about ClickFix attacks is how simple the social engineering is. There’s no phishing email to analyse, no malicious attachment to sandbox. The user is just browsing a normal website and something on the page tells them to paste a command. The command itself looks like gibberish. Most people have no reason to know what rundll32 is or why a website would need them to run it.

ipcrawl

Kind of creepy OSINT-based IP camera crawler. It has 8k+ publicly facing IP cameras that display images and/or video feeds. You can filter across countries, ISP, keywords, and console mode, which looks like Omegle.

🔗 Open Source

BushidoUK/Ransomware-Tool-Matrix & BushidoUK/Ransomware-Vulnerability-Matrix

BushidoToken’s updated Ransomware Tool & Vulnerability matrices from the blog post featured in Threat Landscape above. These are fantastic resources for detection research and creation, especially if you feed your environmental context, ruleset and enrichment from these matrices into an LLM.

atomiczsec/Noradrenaline

Noradrenaline is a set of small offensive shared‑library modules for macOS and Linux meant to be plugged into Poseidon and other post-exploitation agent frameworks. IMHO, this is a great set of capabilities for a detection engineer to test their EDR and detection rules on macOS. I hope someone does Atomic Red Team for macOS soon so this becomes easier and easier!

PaperMtn/claude-enterprise-detections

Andrew Byford’s detection ruleset for their Claude Compliance API research is listed in State of the Art above. It contains the full architecture of detection rules, pipelines, judge integration, and pre-filtering.

assafkip/kipi

Kipi is a self-hosted OSINT platform that uses Claude as its analysis backbone. It’s similar to SpiderFoot in many ways, but I don’t see as many one-off modules or scripts, since it lets Claude make tool calls like API or HTTP requests. The cool part here is how Claude builds relationships via a graph and presents it to you while performing its analysis.

Welcome to Issue #159 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I just got back from a work trip to NY and got to hang out with people in my org who traveled from all over the world (Europe, Asia, and around the US!) Sorry for not putting out an issue last week, but I definitely needed to maximize my time to spend with folks in the office :D

• I hosted an internal security fireside chat where Ayoub Fandi, who runs the GRC Engineer Newsletter, and he crushed it. It was great to hear his perspective on modernizing GRC functions and on the emergence of amazing tech applied to age-old security problems

• It’s almost summer here in New England! I know it’s close because every week, I walk outside with my kids and pick wildflowers. There is almost always a new type of flower blooming every week that we get to bring back home

Sponsor: runZero

Are You Writing Detections for an Attack Surface You Can’t See?

Your detections are only as good as your asset coverage. If runZero isn't in your stack, you're writing rules for an attack surface you can't fully see… and AI-generated exploits are built to find exactly those gaps.

Know every asset, every exposure, every attack path—across IT, OT, IoT, cloud, and mobile. Validate your segmentation before the exploit drops. No agents. No authentication. No appliances.

Try It Free Today

💎 Detection Engineering Gem 💎

Introducing EvidenceForge: Synthetic security logs that don’t look (as) fake by David Bianco

I’ve worked in security product startups for years and have procured vendors and software related to security. The single biggest problem with the procurement process, IMHO, is evaluating the theoretical scenarios in a demo environment to see how accurate they would be in a real scenario. In the ideal state, you turn a product on, and it starts finding the bad stuff, but in my experience, it rarely happens.

The same thing happens with detection rules and capturing a truly malicious, compelling event to guarantee the alert will fire. As a community, we helped alleviate this pain with excellent Purple Teaming tools such as Stratus Red Team and Atomic Red Team, as well as attack-range data, including the Botsv3 dataset. The problem at its core is realism: isolated attack chains work well for sequenced rules, but they don’t take into account the legitimate traffic that can harden the necessary vs sufficient assumptions you need to deploy a rule before it hits prod.

This is where Talos’ EvidenceForge project fills that niche. It’s an LLM-backed synthetic telemetry generation platform that preserves the causal relationships that detection engineers should expect to see during an attack chain. Through realistic scenario generation via prompting, it tries to build scenarios and telemetry that you’d expect to see in a live environment.

EvidenceForge first creates a shared event model to guide synthetic log generation without forcing the user to reason about every downstream format up front. Once generation occurs, it ties scenarios together across host and network telemetry, introduces benign traffic and prerequisite events for the attack, and adds a timing layer. Detection engineers can then use the resulting dataset to validate rule efficacy, and Bianco notes that it can also support threat hunting, analyst training, and incident response exercises.

Projects like this are where I am bullish on agentic technology because they focus on harnessing and acknowledging what does and doesn’t work in model prompting at specific steps. Adding guardrails around the model, such as the Shared Event Model and the realism layer, helps scope the task for the agents, ensuring they don’t make assumptions beyond the prompt's bounds.

🔬 State of the Art

The pressure by Daniel Stenberg

I’ve included a ton of Stenberg’s blog posts on how the cURL team has dealt with surges of vulnerability reports due to LLM use. Many of these posts have shed light for me on how tiring open-source software maintenance work is, just to fix vulnerabilities. The problem in cURL’s case, and I’m sure many others, is the tension between popularity, attention and capacity. The project is already popular and gets lots of attention from software engineers and vulnerability researchers alike, so the volume of bug fixes and vulnerability reports puts a lot of strain on their day-to-day. Add LLMs on top of this; it reads as if it’s unbearable.

This is a sobering and, in my opinion, sad quote from Stenberg. It highlights how underwater he feels while maintaining the cURL project. If I had to relate this, it’s very similar to what we deal with in the security industry. Don’t get it twisted: I think building and maintaining cURL is way more important than my day-to-day in many ways, but the pressure of delivering security initiatives is so much higher than it’s ever been in my career.

Be careful of burnout and take care of yourselves and your loved ones. I think OSS development and security are altruistic in so many ways, but it shouldn’t interfere with your health. I’ll put the cURL donation page link below if you all want to help out a great project.

Donate to cURL

Detecting Tycoon 2FA AiTM attacks across Entra ID and Google Workspace by Samir Bousseaden & Terrance DeJesus

The evolution of phishing has been fascinating to stay on top of since my days of first catching phishing threat actors in 2014. Typically, I’ve associated phishing kits with targeting users to steal passwords and credit cards for fraud. I’m sure most of your friends and family have asked you about spam texts for unpaid tolls, missing packages or a crypto company “threatening” to turn off their accounts. These target the everyday person.

Advanced “attacker-in-the-middle” kits, like Tycoon, target enterprise and cloud accounts. In this post, DeJesus and Bousseaden showcase opportunities for post-phish detection in Tycoon 2FA. Tycoon 2FA is a prolific phishing framework that targets cloud accounts, especially Entra ID and Google Workspace, to provide operators with post-compromise access to sensitive corporate environments. It moves the question from “how do we detect phishing attacks” to “how do we find identity compromise in cloud environments?” Check out the flow below:

It looks eerily similar to how authentication flows, both for OAuth apps and individual users, because the attackers understand the identity model. Hunting for phishing pages and infrastructure is a strong indicator, but when paired with control plane logs from Entra ID and Google Workspace, you can generate high-fidelity alerts.

Device Code Lab (DCL) — Deep Dive into a Device Code Phishing Toolkit by Paul Newton

Continuing the phishing kit theme from Elastic’s post above, Newton dissects the Device Code Lab (DCL) phishing kit to demonstrate its functionality. This is a great piece for comparing and contrasting with other phishing kits. DCL, for example, has unique properties that make it more of a full-service offering, including integration with residential proxy networks, expired domain hunting, and an operator/post-operator model similar to a Cobalt Strike Team Server.

For the most part, the phishing workflow is the same, but uses some Cloudflare Worker shenanigans instead of Socket.io:

There are three interesting features that I listed above:

• Residential proxy integration lets operators route device-code traffic through consumer ISPs rather than obvious cloud ASNs, making sign‑in patterns look much more like real users.

• The expired domain hunting feature continuously discovers and recycles aged domains, so links land on infrastructure with an existing reputational history rather than freshly registered junk.

• The operator / post‑operator model separates the initial phish from the later tenant abuse, giving you two distinct activity phases to hunt for in identity logs, similar to how you’d reason about beacons and operators in Cobalt Strike

Look for post-exploitation activity similar to what Elastic recommended above. These seem to be noisy when they gain access to a tenant and perform enumeration. Look for rogue registered devices as well, because invalidating refresh or persistent tokens can only stop immediate access.

After The GitHub VSCode Supply Chain Attack: Is Your Extension Block Actually Working? by Aman Khurana

This post-incident analysis of the VSCode incident in GitHub by TeamPCP is a clear example of why we need to understand how supply chain tools actually install.

Khurana shows that VS Code uses  marketplace.visualstudio.com  only to resolve extension metadata, then pulls the real VSIX from  gallerycdn.vsassets.io  , Microsoft’s CDN, from which the poisoned Nx Console payload originated. Blocking the Marketplace API felt like an immediate win, but it left the CDN “warehouse” untouched, so anyone who knows the deterministic URL pattern can still fetch and install the extension directly. This is the same pattern for pypi (https://pythonhosted.org) and npm (registry.npmjs.org  and popular 3rd parties like jsdelivr).

This is the pattern you should expect everywhere supply chain software meets cloud: app frontends, APIs, and CDNs can all be different domains, which can mess up detection opportunities if attackers start using legitimate cloud hosting services to blend in.

I’ve joked about this before, but detection engineers are now becoming supply chain security experts. We now need to understand how to map the full software installation flow: which domains resolve metadata, which domains deliver binaries, and which local directories auto‑load content without a visible installer. Otherwise, you are tuning detections against the storefront rather than the delivery path that attackers actually use.

☣️ Threat Landscape

Claude Fable 5 and Claude Mythos 5 by Anthropic

Anthropic released a Mythos-class frontier model dubbed Fable 5. It has extensive capabilities across its usual benchmarks and enhanced capabilities in cybersecurity tasks. The cool thing here is that it’ll route certain prompts to Opus if it deems them unsafe for use. The most random (but most important) benchmark is that it beat Pokémon FireRed only using vision. It’s an 18-second video, so I’m unsure if they just accelerated the playthrough or if it actually went so fast pressing buttons that it did everything in 18 seconds.

Cool down before you install: give new gems a few days to be vetted by Hiroshi Shibata

The RubyGems ecosystem just released a “cooldown” feature for its bundle tool. Effective supply chain attacks, such as those carried out by TeamPCP or DPRK, rely on publishing a malicious package after a CI/CD pipeline or a user has been compromised. Once that package is pushed out, people start installing it, and both the open-source and security communities begin flagging it as malicious.

Cooldowns help combat this type of attack by allowing users to specify a cooldown period during which the package version must remain in the ecosystem before it can be installed. The premise here is that, for example, if you set a cooldown timer for 24 hours, you won’t install new compromised packages.

It’s a great governance control for engineering teams, but given the hyper-configurability that open-source software tools offer teams, it will be harder to pull off in practice. This, IMHO, will work in controlled CI/CD environments, but will likely be bypassed on developer laptops.

21 Zero-Days in FFmpeg by Zhenpeng (Leo) Lin

This blog by the Depthfirst research team shows how much latent risk lies within “plumbing” style open source software. Kind of like what I called out in Stenberg’s cURL blog above, the thankless work that goes into these pieces of software that literally touch our everyday lives is astronomical. So, helping the ecosystem become more secure by performing vulnerability research on these projects is overall helpful, but it still raises questions about the toil on maintainers.

That being said, this blog is a great demonstration on the scale and velocity of vulnerability analysis using agents. Their security agent rescanned a heavily audited FFmpeg tree and still found 21 new zero-days, including heap and stack overflows that had been quietly reachable for 10–20 years in demuxers, depacketizers, and RTP paths.

There’s a great explanation of a vulnerable code path in RTSP processing that potentially leads to remote code execution. All the Depthfirst team needs to do is set up a rogue stream of a period piece movie and/or series, and they can totally pwn me.

Incident Notification to the Maine Attorney General by Meta AI

Meta suffered an AI-related incident in which its “high-touch support” chat agent was abused to grant attackers unauthorized access to social media accounts via password resets. But, my analysis isn’t just about this incident; it’s about how awesome Maine (the state I live in) should be your early warning system.

Basically, when a security breach occurs at companies like Meta AI, they go through a notification process to make sure affected users are aware of the incident. This is especially important when trying to comply with federal and state laws. Maine has some of the strictest requirements for breach notification if any Mainer (yep, that's what we call ourselves) is affected by a breach.

Here’s what Perplexity cooked up as it did some research on this for me:

• Companies must notify affected Maine residents and the Attorney General “as expediently as possible and without unreasonable delay,” which in practice means they cannot sit on a breach for months while they polish a narrative.

• If more than 1,000 people are notified, they also have to notify consumer reporting agencies and file a detailed notice with the AG that becomes part of a public record, even if only a handful of those people actually live in Maine.

• Notice has to go out as soon as law enforcement says it will not compromise an investigation, with guidance converging on roughly a 30‑day outer bound from discovery, so Maine filings often appear before a company’s national comms and investor disclosures are fully baked.

If you want a live view of the feed, here’s the “Breach Portal” for Maine:

https://www.maine.gov/agviewer/content/ag/985235c7-cb95-4be2-8792-a1252b4f8318/list.html

🔗 Open Source

Cisco-Talos/EvidenceForge

GitHub link to Bianco’s EvidenceForge project listed in the Gem above. Check out the AGENTS.md file since this is run as a skill within Claude / Codex.

tjnull/cygor

Open-source, Python-based asset discovery framework. It handles host discovery, port scanning, service fingerprinting, and enrichment in a single path, then organizes the results into a single web app. Good fit for attack surface inventory, lab mapping, and quick scoping when you need a clean picture of a subnet or environment.

cjpais/Handy

Handy is essentially a push-to-talk assistant for transcribing your speech into text. It uses local models for the transcription itself. Good for note-taking, but might be super interesting for prompting your coding agent so you can be totally hands-off keyboard.

CarterPerez-dev/exs-cyberjob-scraper

Elixir-based scraper that pulls cybersecurity job postings, filters out the generic “IT” noise, and surfaces which certs show up most often in real roles. If you mentor juniors or run hiring, this is a handy way to sanity‑check which certification signals actually appear in current job ads instead of guessing based on vendor marketing.

Welcome to Issue #158 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• Over the weekend, I was clearing out some brush in the woods behind my house since we want to create a play area for the kids. Once we get the deadwood and sticks away, we want to clear some plants as well. I pulled out my phone and scanned the plants to identify them, just in case it was poison ivy, and lo and behold, we have blueberry bushes everywhere! And I mean everywhere, I’ve asked my neighbors to come dig some up if they want some for their yards.

  

• I’ll be in NY next week for work, and will be attending Sprawl on Thursday. If any readers are attending, let me know!

• Lastly, my org at Datadog just opened up two engineering manager positions for our detection engineering team. We are expanding our detection engineering here at Datadog and would love to have some folks come in and help shape it in the age of AI. U.S. based applicants at this time :).

  Apply Here!

💎 Detection Engineering Gem 💎

Perplexity Is Open-Sourcing Bumblebee by Perplexity

Perplexity, a major AI company in the AI search space, released a Go-based secret scanner dubbed Bumblebee. Bumblebee is a secrets scanner, but it has a much more focused approach to targeting open-source software and 3rd-party vendor supply chain attacks. Every week, I read and likely link threat landscape stories involving an open-source supply chain attack. Perplexity deployed this tool in a clever framework to quickly orient to emerging attacks and check their exposure on developer laptops:

I’m sure Perplexity uses its own product to create its “threat intelligence feeds” listed at the top left, then passes them to its Perplexity Computer Agent. The agent collects as much context as possible around the attack and drafts a pull request to their exposure catalog, which is essentially a database of known malicious packages derived from their feeds.

The updated catalog kicks off a separate workflow to scan Perplexity’s fleet of laptops for emerging threats, while continuous scans are run on known-compromised packages from previous campaigns. The “worst case” scenario is finding successful compromises of devices, but the beautiful thing about this is the reliability of the build-and-scan pipeline, scouring for new campaigns. The package inventory and audit logs are excellent for false-negative analysis, especially in scenarios where a more subtle campaign that doesn’t perform smash-and-grab attacks can help responders with threat hunting.

I’ll say this again, as a word to all my readers who work in detection & response: please upskill in software supply chain security! We need to understand how these campaigns operate so we can hook into CI/CD pipelines to generate detections and audit logs for hunting.

🔬 State of the Art

Built a SOC from scratch with no prior SOC experience (Reddit Post) by After_Marsupial_3531

I am seeing more and more discussions on Reddit about security operations programs, and this one specifically impressed me because the author rolled out a full SIEM stack as an apprentice at an MSSP. They essentially have three people in their team: themselves, a CISO, and a salesperson. What they managed to do with zero experience and launching an MSSP service is nothing short of a miracle:

I have zero experience in a mature SOC — and neither does anyone else on the team. I’m figuring things out as I go, and I’m not always sure whether what I’m doing is actually sound or consistent with industry standards.

When I hear about stories like this, I immediately jump to the worst-case scenario. No experience in a SOC, fresh out of an apprenticeship and architecting a competitive managed service seems like a recipe for disaster. But they managed to launch it with real customers, primarily looking at EDR & M365 telemetry. Elastic Security was their stack of choice, probably due to its simplicity of out-of-the-box rules and its being an open architecture.

The resulting discussion is great, as they were looking for feedback on their deployment. There is definitely some memeing in the comments, with good nuggets and insights for the author, such as finding a way to create a crown jewels or critical assets list and prioritizing high-risk scenarios a small business would face.

Project Glasswing: An initial update by Anthropic

I don’t know what it is about this year’s developing AI & security space, but 1 month feels like 6 months in terms of technology acceleration. Anthropic just gave their 1-month update for Project Glasswing, and the results are impressive. They released Mythos to a limited set of company partners they deemed to be Internet-essential software companies, and also ran it on 1,000+ open-source projects. The results speak for themselves, but there are some nuances:

A lot of work goes into verifying vulnerabilities, both their presence and their reproducibility. Anthropic spent a lot of human energy, rightly so, to make sure that these vulnerabilities wouldn’t waste a human’s time. They employed independent security firms to help verify their findings and touted a 90%+ true positive rate. The nuance here is the patch cycle itself.

I do think this year is a landmark year for rapid vulnerability discovery and disclosure, but we are certainly not close to it being the year of rapid vulnerability patching. The bottleneck is the process of fixing, testing, merging, and deploying patches at scale. I hope Anthropic continues to invest in this space and moves towards rapid deployment of patches. Our industry, conferences, and community reveres vulnerabilities as the pinnacle of achievement for a researcher.

Maybe it’s my blue team bias, but the pinnacle achievement for me is protecting others. I’m unsure if more disclosed vulnerabilities protect others, because the externalities around patch time, exploit time, and volume may make our situation much worse.

If you haven’t read the Project Glasswing analysis above before reading this one, please do, because this episode is very relevant to my analysis. Shocker, it tends to agree with JAGS and the crew here :).

What I love about this episode is the sober grounding around the concept of “10xing vulnerabilities” due to these models. There is a lot of talk about the volume of patching from early-access vendors in Glasswing, but this tends to overlook the critical part: customers actually applying the patch!

Much like I said in my Glasswing update, I will be impressed if we can have companies that build and sell appliances, IoT devices, or even endpoints take a more opinionated approach to forcing patches. JAGS gave some ideas around this, such as giving a window for people to apply before a vendor forces the update. This has its risks, especially around governance, customer success, and usability, but if we want to tout the success of AI in security, then the boring part of patching needs to be solved just as much as the exciting part of vulnerability discovery.

Manage extensions in enterprise environments by Microsoft Visual Studio (Documentation)

Following GitHub’s disclosed security breach, someone on Reddit posted the documentation for managing, securing, and deploying Microsoft Visual Studio extensions. It was released five days ago, so it’s likely this was posted in response to the VSCode extension attack on GitHub, which resulted in its public disclosure. For the most part, these protections are great, but I worry about a few things.

First, it assumes you have a list of all extensions that every developer uses, that you can accept and apply a deny-all afterward. For those who work with a large engineering team: good luck with that! Second, trusted publisher allow lists wouldn’t have stopped the Tanstack attack since the trusted publisher itself was compromised. Third, besides the private registry, I wish there was a native way to grab an inventory of these extensions for analysis, much like what Bumblebee from Perplexity does.

Microsoft be Microsoftin’.

☣️ Threat Landscape

The Future and Past of Residential Proxies by Qurium

I learned the term “botnet” in my first year of college in 2008. The concept seemed cool: you infect a device, build a Rolodex of infected devices, and collect them like Pokémon cards. I never really understood the financial appeal because everything leading up to the massive DDoS botnets, such as Mirai, was mostly around selling access to perform DDoS attacks to make a statement. The recent development of residential proxies in the last two years, though, exposes a much more lucrative financial model and a more serious threat than DDoS attacks.

Qurium’s piece here eloquently lays out the history of this evolution and gives great context for those who worry about residential proxy abuse. It’s worth reading less for the individual provider callouts and more for the ecosystem shape. The interesting story is how Android supply-chain compromise, proxy SDKs, DDoS botnets, and commercial residential proxy markets start to blur together.

Qurium is on the receiving end of this abuse and has been mapping the providers, botnets, Android supply chain compromise, and proxy monetization layers that enable it. The useful takeaway is not that every residential proxy provider is malicious. It is that the line between “commercial proxy service,” “compromised device pool,” and “botnet infrastructure” is getting harder to see from the outside.

Lawmakers Demand Answers as CISA Tries to Contain Data Leak by Brian Krebs

Krebs provided a follow-up update on the initial break of his CISA Leak story, in which he obtained letters from lawmakers asking CISA to answer questions about how the leak happened. He posted a picture from one of the letters, and it looks like Krebs' story was the first reference in their letter. I wonder if he prints these out and frames them?

With CISA funding and workforce being slashed, as Krebs points out, it's hard to understand whether this leak happened due to reduced security controls, a burnt-out staff, or discontent. CISA took over a week to rotate some credentials, including an RSA private key, after having researchers from Trufflehog and GitGuardian reaching out multiple times.

Cybercriminal VPN used by ransomware actors dismantled in global crackdown by Europol

Europol seized servers and assets, and even interviewed an administrator of “First VPN”, a residential proxy network. These networks allow users to purchase access to typically unwitting endpoints, such as cellphones, routers, and VPSes, to conceal malicious activity and make it much harder for defenders to distinguish legitimate IP addresses from malicious ones.

I’ve always been impressed by these joint operations, especially those led by Europol. They have to coordinate across countries and jurisdictions, and they manage to navigate the bureaucracies of these agencies to execute coordinated takedowns that can include seizing physical servers, domains, and other infrastructure.

Supply Chain Attack Targets Laravel-Lang Packages with Credential Stealer by Iyas Makari

Aikido’s Iyas Makari publishes their research on a supply-chain attack targeting Laravel. Laravel is a PHP framework used by developers to build and manage PHP applications and is widely used by websites worldwide. There are two interesting findings from this report I want to call out:

• I am worried that supply chain attacks affecting PHP applications are more likely to reach VPS servers themselves. This can open the door for more Mirai/Kimwolf-style residential proxy botnet infections, where the goal is to monetize access to compute rather than keys

• The actor pushed malicious packages and pointed them to an orphaned commit on a fork, making it much harder to detect on the main repository

I don’t think bullet 1 is happening yet, as Makari reverse-engineered the infostealer itself, which performs typical infostealer-like things such as swiping secrets and API keys. But those secrets do exist in deployed web applications, and not to sound biased, but I don’t ever think of a PHP website hosted on a VPS is as “secure” compared to what we are used to in SaaS land :).

🔗 Open Source

perplexityai/bumblebee

Perplexity’s package scanning and supply chain security project linked from the Gem above. It’s a single Go binary you can run on your endpoints, typically paired with an MDM solution. It’ll scan for malicious packages and generate alerts, audit logs and build asset inventories.

gadievron/honeyslop

Much like the prompt injections you see people post about on LinkedIn, where a description in their profile has a prompt that an AI recruiter uses when sending out messages, honeyslop does this but for codebases. It’ll add prompt injections as code itself, where AI vulnerability harnesses will read and report on fictitious vulnerabilities so you know when you receive an AI slop report.

rootsecdev/relay_bible

GitHub native course for readers to learn about NTLM-relay style attacks, tools and frameworks. It has several sections that first introduce what an NTLM relay attack is, and then expands out to concepts like coercion attacks, different tools and advanced techniques.

grepstrength/malsnitch

malsnitch is a neat malware analysis helper that looks for secrets inside malware artifacts. It takes strings output, FLOSS JSON, or Binary Ninja exports and pulls out things like C2 credentials, crypto keys, API tokens, Discord webhooks, Telegram bot tokens, and hardcoded exfil configs. This is nice if you are tracking malware families that have hardcoded C2 servers that frequently change so you can update your blocking lists. These pivots can help uncover additional infrastructure, or map out existing infrastructure for your own security research.

Welcome to Issue #157 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• Spring in Maine is finally here, and it feels like a Disney movie at the Allen household with all of the animals coming out. We have two bird nests, a family of deer, and now a woodchuck named Chippy prancing around

• Speaking of Maine, I had the most Maine dinner with some friends the other night. They hunted a Moose and cooked Moose steaks for the family. Afterward, we had dessert with maple syrup tapped from trees in their yard. Moose was not that gamey, and I’d try it again!

• Sabres lost :( But I’m super excited to watch this young team in the coming years, so if you are a hockey fan, don’t tease me too much because I’m still hurting from the loss

Sponsor: Adaptive Security

The best threat intelligence is generated by your coworkers

Security Operations teams rely heavily on detection rules and threat intelligence to stay ahead of threats. But, the AI of today focuses on social engineering your coworkers, such as deep fake interviews, voice phishing, and impersonating executives. Training your employees to spot and report AI attacks can provide valuable signals that no security product can match.

Adaptive trains your organization to spot, report, and stop these AI threats before they reach your perimeter.

Take A Tour

💎 Detection Engineering Gem 💎

Shai-Hulud Goes Open Source by Ryan Simon, Sebastian Obregoso and Greg Foss

~ Note, Datadog is my current place of employment, and the authors here are my colleagues ~

I’ve covered several supply chain attacks linked to TeamPCP and one of their worm-based toolsets, Shai-Hulud. This week, GitHub repositories popped up and were quickly taken down, hosting the potential source code for Shai-Hulud.
The team here did a deep dive on its architecture, how it deploys loaders and malware, its C2 infrastructure, and its credential exfiltration techniques.

The Mutator step, which enables Shai-Hulud’s propagation and worm-like capabilities, is super interesting. There are three propagation mechanisms: one I was familiar with, npm credential publishing, and two I wasn’t familiar with, cache poisoning and developer environment poisoning. Every stolen secret can be used to propagate malware across various developer environments and IDEs, such as Claude Code and VSCode.

The two npm-related techniques rely on insecure configurations for publishing npm packages. For example, if you store npm publishing tokens in your GitHub Actions pipeline, it’ll automatically download the latest version of your release, increment the semver to make it the latest branch, then deploy it, causing the post-install hooks to run, which contain the malware.

“Leaks” of malware source code like this help provide insight into how threat actors think about exploiting misconfigurations and security gaps in their targets. It’s hard to say whether this is the “real one”, but there is threat intelligence floating around social media that TeamPCP & cybercriminal forums are teaming up for a Supply Chain Competition to see who pulls off the most impactful breach with Shai-Hulud.

🔬 State of the Art

Wiring Live Threat Intel into Sigma Detection with Dynamic Pipelines by Mostafa Moradian

Most SIEMs let you manage threat intelligence as lookup tables or reference lists that detection rules can query against. A rule checks a destination IP against a known C2 list; the list is updated by a feed; the rule stays the same. It is one of the more practical parts of how enterprise detection pipelines handle the IOC lifecycle. It’s also VERY hard to manage, and without understanding the underlying engineering required to handle dynamic tables, you can produce stale IOCs for your detections, or worse, miss emerging IOCs and fall victim to false negatives.

Moradian has been building toward the same capability in his RSigma project. I first covered the project in #149, when it was a Rust CLI for evaluating Sigma rules against JSON logs without any infrastructure. By #154, it had grown into a streaming daemon with NATS and HTTP log inputs, stateful correlation windows, and Prometheus metrics.

The latest addition here is dynamic pipelines: instead of hardcoding IOC values in rule YAML, you declare external sources in the pipeline config, and RSigma fetches and injects them at evaluation time. This works very similarly to how I’ve seen SIEMs implement threat intelligence pipelines, but since it’s RSigma, it’s self-contained within its ecosystem.

The practical demo is a firewall detection rule with a single field: DestinationIp: “%c2_ips%”. The pipeline pulls the Feodo Tracker C2 blocklist, a public JSON feed covering Emotet, Dridex, TrickBot, and QakBot updated every five minutes, extracts the IP array, and expands that placeholder before the rule evaluates. If the feed is unreachable, RSigma falls back to the last cached data so you do not lose coverage during a transient outage. The rule author never touches an IOC value again.

I thought this was a neat video by the Anthropic team, featuring a friend of the newsletter, Jackie Bow, on how their internal detection & response team uses Claude! They built a tool called CLUE, which they first featured at BSides SF in 2025. It looks similar to a co-pilot tool from AI SOC vendors, but it's their own internal version. I think it helps make the case for those who choose to go build versus buy, by offering control over which features go in and out of the tool, an integration catalog focused on systems you deploy for your organization, and a tighter feedback loop.

The case against is a capacity-and-cost problem. It is attractive to buy AI SOC-like tools if you are budget-constrained or don’t have the talent to build and maintain one.

Ok, a lot of media on this issue, but hear me out. Odd Lots is a Top 3 podcast for me, and its focus is Finance topics. They bring on amazing guests that talk about everything from the economics of milk to tarriff refunds at shipping ports and even cybersecurity.

I chose this episode because its about the history and economics of deep-sea cables, with a particular focus on Internet cabling. We seriously don’t appreciate how complicated and opaque this process is for the end user. So, if you want to learn about Internet-scale issues, you must learn about the physical nature of operating the Internet!

The Ultimate Guide to Detection Engineering with Censys by Alex Gartner

In this post, Censys detection engineer and security researcher Alex Gartner helps readers understand how to integrate detection engineering concepts into Censys’ internet-wide scanning data. This is especially interesting to me in a few ways:

• I’ve rarely read posts on how to integrate Internet scan data as correlation rules with traditional telemetry from the endpoint or network

• Understanding how attacker infrastructure can be fingerprinted is a valuable skill in itself; it’s not all just IP address and domain matching

• Speaking of IP addresses: threat intelligence is much more useful when it’s clustered and used as an enrichment, and we leave out a lot of interesting threat intelligence indicators

I’ve been a big fan of Censys since they started their company several years ago. And luckily, many of the concepts Gartner discusses are available in community edition accounts. I typically don’t link to vendor blogs that talk about their products unless they offer free or community-edition access, so you can try these concepts out by signing up.

☣️ Threat Landscape

Grafana, an observability company with a large open-source following, disclosed a security breach in which attackers obtained an administrative-level GitHub access token. They don’t specifically say if it’s a long-lived token, which GitHub has urged customers to use, but it’s likely if it survived a CI/CD build run. The group behind the incident, Coinbase Cartel, also likely used a pwn request to take the token and subsequently exfiltrate their entire codebase.

The group tried to extort Grafana with a ransom demand, but Grafana said in their tweet that they refused to pay. This is the right approach, since paying doesn’t guarantee that the thiefs won’t sell it to someone else.

Oh look, a VPS I purchased 8 months ago. by Jonathan Canlas

In this post, Canlas, a detection engineer, stumbled upon a VPS they had purchased several months ago and had accidentally left running. They turned it into a honeypot to understand what kinds of attacks it attracts, using subsequent telemetry to learn more about the research and detection workflow. I appreciated reading this post because most honeypot stories I read follow a formula: a lab that deploys a honeypot. This one, in particular, offers much more insight into Canlas’ experience as a detection engineer and how they approached the deployment through a lens of hypothesis-driven research.

Novel Evilginx Frontend - Lowering the barrier for token theft reuse by Paul Newton

Evilginx is a legitimate open-source framework that helps red-teamers build and deploy phishing pages with easy-to-use templates. There is a pro version where the company behind Evilginx vets prospects to ensure they aren’t evil, but the open-source version gets a lot of traction from bad guys. In this post, Newton was hunting for M365-related phishing kits and found a cleverly built M365-themed management interface. IMHO, it makes the post way more interesting since it’s about this peculiar management tool.

The potential threat actor built this front end to manage stolen M365 tokens at scale. They can switch between victim tokens within a single interface, import stolen tokens from their Telegram bot, and execute attacks with a single click.

A single stolen token grants the operator access to several products in the M365 suite, up to and including pivoting to take over the Entra environment.

When AI doomers or marketers post about the onset of dangerous, vibe-coded malware at scale, I roll my eyes because most malware is a commodity at this point. Evilginx is so good, so why switch how it works? I worry more about the organizational scale that vibe-coding offers threat actors.

Before Claude Code, creating an interface like this would have taken months, either with a professional front-end developer or with someone who spent a lot of time learning front-end coding. Front-end coding with coding agents is AWESOME, so it’s now much more accessible for people to build things like this without spending their capacity on learning.

Project Glasswing: what Mythos showed us by Grant Bourzikas

Cloudflare ran Mythos Preview on more than 50 of its own repositories as part of Project Glasswing and published the results. Much like what Anthropic noted about Mythos in their Glasswing blog, Cloudflare said it was excellent at generating exploit chains and generating proof, via a PoC, that it found a bug. The harness architecture they built around it is worth reading too: they created narrowly scoped parallel agents, each focused on a single attack class, and then used an independent adversarial agent whose only job is to disprove the original finding.

Nothing in here really surprised me regarding the applicability of Mythos to real-world code environments. An agent is only as good as its harness, and although Mythos has fewer guardrails and is tuned for security findings, it still requires work to operationalize.

🔗 Open Source

vinsk0h/KQLab

KQLab is a self-hosted platform for storing, protecting, and indexing KQL rules. It has an impressive full-stack architecture and is built to provide strong authentication and verification measures for users, and even includes RBAC capabilities to scope certain rule sets to specific teams.

OWASP/DockSec

DockSec is an OWASP Incubator project that builds a vulnerability scanning report from several open-source tools and generates human-readable reporting leveraging an LLM harness. It’s smart enough to prioritize vulnerabilities based more on CVSS scores, and you can stick it in your CI/CD pipeline so it can recommend fixes that developers understand, rather than a massive findings report.

phylecorp/gabo

Gabo is a Nebula:Fog hackathon project built for threat intelligence analysts. It uses the CIA’s Structured Analytical Techniques playbook to provide users a tool where they can ask it questions, and it’ll provide rigorous feedback based on their analysis. It has an MCP server as well, so you can point your local coding agent or stack at it.

microsoft/AntiSSRF

AntiSSRF is a security component library from Microsoft that helps developers deploy apps without worrying about the underlying techniques used to prevent SSRF attacks. It has a robust URL parsing library, which are always a headache to work with, to validate incoming HTTP requests and reject any that look like SSRF attacks.

Welcome to Issue #156 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• Another Allen first! My daughter had her first T-Ball game and after trepidation in practice, she surprised all of us with some very competitive plays and dogpiled on the baseball with other 4-5 year olds

• I booked my hacker summer camp Vegas travel so I’ll see you all at BlackHat & DEFCON. I’ll be trying to host a Detect & Response Happy Hour again, so stay tuned. If any current or future sponsors want to do an event, or sponsor the Happy Hour, feel free to reach out to me! techy@detectionengineering.net

• I’m starting to kick the tires on Codex and have been impressed with it so far. I’ve also had an itch to resurrect my OpenClaw instance for more malware research so wish me luck with my wallet :)

• The 3rd edition of Datadog Detect is next week! Datadog Detect is a FREE, fully-online security operations focused event hosted by the Security Research team here at Datadog. We’ll have three presentations from practitioners and a short panel discussion afterward. The first two were excellent and I hope to see you all on this one. Checkout the schedule and the registration link below!

  Register for Datadog Detect

💎 Detection Engineering Gem 💎

Coverage Is Not Detection by Gabriel Abdelgawa

I read a ton of blogs and thought pieces about detection coverage. MITRE ATT&CK is the industry standard for explaining this concept, and it has its uses. It answers the question “am I covered?” and is a useful tool for communicating to others that you don’t have a visibility or detection gap. It has also been abused by product marketing, especially when vendors claim 100% coverage or more coverage than their competitors. Detection engineers know this, but they don’t necessarily know why it’s a double-edged sword, beyond the nuances of rules and how they can be bypassed.

Abdelgawa’s blog post on this is the cleanest and most concise description of the inherent problem with detection coverage as a metric. The problem is that they are defensible because it’s a number you are communicating. They aptly reference a concept from Theodore Porter on mechanical objectivity. The idea is that quantifying a problem, such as with a percentage, is harder to challenge than a qualitative approach. Presenting 90% coverage on an ATT&CK heatmap biases the audience because they assume the measurement that yielded that 90% used sound methodologies.

We all know that’s not true :).

Abdelgawa cites several examples within detection scenarios where coverage starts to fail. The examples are rooted in three categories:

• Depth: The idea of coverage is binary, so if you detect obfuscated PowerShell usage, and you have one rule, are you covered?

• Relevance: What is relevant to your environment, and are you assigning equal weight to a Windows technique if you run a full macOS shop?

• Independence: Focusing a detection opportunity on a singular telemetry source can fail you if that telemetry source, such as EDR, doesn’t generate the correct telemetry or find the activity at all

I’ve argued ad nauseam about coverage to folks, and it’s never the detection engineers who disagree with me. It’s mostly leaders and vendors. And TBH, I don’t blame leaders. We need a better way to measure the readiness of detection rules and security incidents in general. So, use a number as an anchor point, but make sure you provide context and methodology so it can be challenged.

🔬 State of the Art

ATT&CK Coverage Ratio Evaluation (ACRE) by Jordan Anderson

This blog release timing is excellent with Abdelgawa’s Gem post above! The gem above diagnoses why coverage maps fail as a measurement device, and Anderson's blog is essentially the constructive counterpart. Anderson’s ACRE measurement fairly well addresses the two problems of relevance and independence in detection coverage, and, self-admittedly, depth is much harder to solve.

The “ratio” in ACRE helps teams derive a denominator of techniques more likely to be mapped to your environment, forming a subset of the full MITRE ATT&CK map. They leverage MITRE’s CAR Catalog, a set of rules compiled across several vendors, to build a realistic roadmap of detections you should implement to achieve coverage. This can fail, IMHO, if this catalog is not kept up to date, but just building the roadmap for coverage alone is worth using CAR.

ACRE also solves the “everything carries the same weight” problem by adding a weighting mechanism. Techniques that live in the middle of ATT&CK, such as persistence, should have more telemetry and detection opportunities than those in the beginning and end stages. So, when ACRE computes the full score, it assigns greater weight to coverage for these middle-stage techniques and then splits them out by OS. You then have a much cleaner numerical score on coverage and weights within each environment.

Could your choice of metrics be harming your SOC? by Dave Chismon

As we move downstream from detection metrics to SOC metrics as a whole, we return to the question of how we measure success in security and the pitfall of mechanical objectivity. Chismon rightly calls out that efficacy in security operations is rooted in an IT-centric productivity culture. This focuses on units-of-work measurements, such as the number of tickets processed (alerts triaged), meeting KPIs or SLAs (mean time to touch an alert), and other human-based tasks.

Chismon calls out four metrics that fall into the mechanical objectivity trap: the number of tickets processed, the time to close a ticket, the number of detection rules, and the volume of logs. We tend to take metrics like this and optimize around them because we assume they measure SOC performance. But, much like the pieces above by Anderson and Abdelgawa, they don’t answer the question of “how ready are we for a security incident?”

Chismon suggests seven metrics at the end of the post, and what I love about these is that they are a mix of metrics we know and love, and human-based metrics. A successful SOC, for example, is one full of people who are satisfied with their jobs and engaged, so why not measure that? The same thing applies to engagement beyond the security team and to making sure the analysts know the environment they are trying to protect.

Appearing Productive in The Workplace by No One’s Happy

This blog isn’t necessarily detection or even security-related, but it’s especially relevant for all of us operating in a coding-agent world where everyone seems to be building code. It offers a meek look at what coding agents can do well for anyone, but also what they don’t do for those who understand the underlying technologies, idiomatic techniques, and gotcha’s associated with software engineering. It plays nicely into the security-relevant piece on the cognitive rust belt by St-Meyers in a previous issue.

These risks are real, and people are concerned about untrained people writing software. Putting the security piece aside, the reliability of these systems and the likelihood of strange architectures that don’t fit practices built and honed over the last 30 years mean they risk being dumped on the software teams who did not build them.

On the contrary, it does throw the baby out with the bathwater, in the sense that I don’t see anything about how this technology can be used for good to help these folks. I tried writing about this in my piece about agentic coding x security below:

Inadvertent Injections by sud0woodo

In this post, sud0woodo hunted for PoisonedRefresh samples first discovered and posted by ESET on Twitter. PoisonedRefresh is a piece of malware that targets F5 devices and creates an implant within the Apache process to execute a webshell. The x64 variant they found on VirusTotal is more portable than the original sample and includes a peculiar design decision that could potentially expose researchers to accidentally compromising a device.

The implant hooks read() and runs a KMP string search against every GET and POST request. The upside of KMP is that the magic trigger can live anywhere in the request, such as a cookie value, a referrer, or a JWT payload, which makes it much harder to fingerprint the implant from the outside.

According to sud0woodo, if security companies or researchers are actively scanning for this variant and their probes carry the magic string, they can trigger the webshell injection on a box that the operator has not yet activated. The author calls this an inadvertent injection, and it can mess up forensic investigations and create false alarms for those hunting for these payloads in their own environments.

☣️ Threat Landscape

Where Have All the Complex Windows Malware and Their Analyses Gone? by R136a1

I remember when Mandiant first released its APT1 report and how blown away I was by the technical aptitude of both the researchers and the threat actors themselves. I was just out of college, and after reading it, I knew I wanted to pursue threat research as my career. Windows was the operating system used to build malware because of Microsoft’s market share, the lack of cloud computing at the time, and the lack of commoditization of attack tools and security knowledge.

Fast forward 14 years (!) later, and the industry and landscape have changed so much that reports like this no longer come out. It’s not for the lack of effort, as R136a1 puts it, but more so several factors around access to intelligence, the advent of GitHub and open-source post-exploitation toolsets, and the cloud and SaaS apps changing the focus of threat actors. Social media, IMHO, plays a huge part here too, because we can all build independent followings without having to worry about throwing time and money into marketing our research.

dirtyfrag by V4bel

Dirtyfrag is a Linux kernel LPE vulnerability that draws inspiration from DirtyPipe. It’s a reliable exploit in the sense that it doesn’t cause kernel panics, and you don’t need to rely on any time windows or race conditions. It relies on exploiting page caches in the Linux kernel, which are privileged kernel memory that help optimize memory reads and writes by preventing repeated disk access.

In DirtyPipe, you can achieve LPE by timing a splice() syscall to gain arbitrary write. Dirtyfrag achieves the same outcome by corrupting a pointer in sk_buff, the Linux kernel’s core network packet structure. The frag member of sk_buff holds a reference to a page-cache page used to build packet fragments.

By corrupting that reference to point to a read-only page backing a privileged file, you get an arbitrary 4-byte write through the network path, allowing you to write suid binaries, overwrite entries in a password file, or do some fun container escape shenanigans.

Mini Shai-Hulud Is Back: npm Worm Hits over 160 Packages, including Mistral and Tanstack by Raphael Silva

The Aikido Research team uncovered a new campaign Shai-Hulud targeting several npm packages, with Tanstack and Mistral’s npm packages being the more popular packages hit inside the campaign. Tanstack’s post-mortem describes a PwnRequest style initial access vector, which then pivoted into their CI/CD pipeline. They abused the lack of trusted publishing inside their environment, and began propagating 80+ malicious versions into the ecosystem that performed the same worm-like attacks that looked like it eventually hit Mistral.

When prompts become shells: RCE vulnerabilities in AI agent frameworks by Uri Oren, Amit Eliahu and Dor Edry

This is an interesting vulnerability research piece that examines how AI agent applications leveraging orchestration layers, such as LangChain, can be vulnerable to traditional command injection and remote code execution. Much like the well-understood attacks against web applications that exploit code paths leading to unsafe deserialization or risky eval() calls, popular AI agent libraries can have the same vulnerable code paths.

Microsoft researchers Oren, Eliahu, and Edry demonstrate this vulnerable code path within Microsoft’s Semantic Kernel project. In CVE-2026-26030, the In-Memory Vector Store feature passes AI model output directly into a Python eval() call to filter search results. They circumvented some filtering logic in the path to break out and achieve a shell.

The second vulnerability, CVE-2026-25592, abuses a helper function that was accidentally exposed to the model as a callable tool. The localFilePath parameter, which controls where the file is written on the host filesystem, lacked validation. The attack was unique here: they generated the payload in the sandbox, then made a tool call to achieve host compromise.

Student’s hack prompts THSRC review by Shelley Shan (Taipei Times)

A student in Taiwan used a software-defined radio to analyze, reverse engineer, and replay signals on Taiwan High Speed Rail’s radio communication system. They sent a spoofed general alarm to the control center that halted trains for 20 minutes. Police found nearly a dozen two-way radios at his residence and confirmed he had also cracked into the New Taipei City Fire Department and Taoyuan Airport MRT frequencies.

I’m unsure what the motivation for the hacks here were other than morbid curiosity and the hacker mindset. I do think this points out several security flaws in Taiwan’s rail system, so they could have done some good and tried to report this. It makes it particularly sensitive as the tensions in Taiwan are higher than ever with PRC, and a “hack” causing physical consequences makes the government particularly sensitive to these types of crimes.

🔗 Open Source

ddamenova/IRQL - Incident Response Query Language

IRQL is a domain specific language that sits on top of KQL to help alleviate both the barrier-to-entry of running KQL and to reduce the complexities of large KQL queries. Analysts use 5 primary functions that help build queries: selectors, extractors, enrichers, graph-listed variants and external enrichments. These look to be re-usable components and idiomatic queries found in KQL functions, and the example KQL <> IRQL query shows an impressive transformation that looks much cleaner and easier to read.

microsoft/EventLogExpert

Windows Event Log viewer built by Microsoft themselves! It looks like a Wireshark but for Event Log analysis, which is nice because parsing these logs does require some expertise not only with the structure of EventLogs but some PowerShell kung-fu to correctly parse them.

Mihir-Choudhary/EventHawk

EventHawk’s latest release has some good timing with Microsoft’s EventLogExpert listed right above this entry. EventHawk is also a Windows Event Log parsing tool, but it has a lot more bells and whistles and a much cleaner user interface for analysis. I love the ATT&CK integration module, so it provides a much more enriched view into EventLogs both for quick analysis and for learning.

beenuar/AiSOC

Self-contained AI SOC you can run locally or on a hosted cloud environment such as Fly, AWS or K8s. It ships an MCP server and exposed tools to call the models and perform the translation layer. These repositories generally look like lab environments or someone’s research project, but this is the first one I’ve found that has a seriously impressive amount of features, toolsets, integrations and a healthy development community.

Welcome to Issue #155 of Detection Engineering Weekly!

This week, I’m resurfacing some of my favorite Gems over the last five months. We’ll be back to our regularly scheduled newsletter starting next week!

✍️ Musings from the life of Zack:

• I’ve officially christened yard work shoes with an old pair of white sneakers. I just need some cargo pants, a cell phone clip attached to my hip, and a Bluetooth headset to really complete the outfit

• I saw my friends playing Slay the Spire 2 on Steam, so I got it, and it’s so addictive! You know it’s bad when you think about game mechanics on your drive to the gym or between work meetings

• Proud Dad moment: my daughter had her first-ever ballet recital, and she crushed it. She was in an auditorium with over 100 people, and she was in the youngest age group. She was all smiles and even found us in the crowd and waved

Detection Pipeline Maturity Model by Scott Plastine

I’m a huge fan of maturity models, and in the early days of my writing, I frequently referenced the work of Haider Dost and Kyle Bailey when discussing the maturity of detection engineering programs. As this space matured, technology matured with it, and we now have complex systems within each part of the Detection Engineering Lifecycle. So, to me, it makes sense that we now have folks like Plastine helping us understand what it means to measure the maturity of a Detection Pipeline.

Plastine outlines six different levels of maturity, starting with a classic favorite, no maturity! This involves having a security tool stack with no centralization, and analysts have dozens to hundreds of Google Chrome tabs open which gives me anxiety. The fundamental issues Plastine outlines and continues to improve here include:

• Several security tools with their own alerting and detection systems

• The need to log into and investigate each alert on each individual tool, so managing screen sprawl

• The analyst manually building cases in some case management or ticketing tool, such as JIRA or ServiceNow

The next maturity step, Basic, addresses some of these issues by essentially placing the Case Management tool between the tools and the analyst, rather than being out of band. As maturity levels progress, so does the architecture of this setup. For example, the “Standard+” architecture has a much saner pipeline setup:

The cool part at this point in the maturity journey is switching from architecture improvements to more advanced concepts in the analytics platform. Custom telemetry, log normalization, and a risk-based alerting engine ideally surface only relevant alerts and reduce false positives. Teams begin to build composite rules, leveraging commercial detections alongside their own internal detection and risk alerting systems, and they all take advantage of learning from their data to inform their rule sets, not just their environment.

This diagram drove it home for me, and became my favorite:

As you progress through maturity, the trap teams fall into is more rules is better. I think the measure of a Leading detection function is reducing rule count thereby reducing the complexity of managing rule sprawl.

Plastine posits that this can be achieved by using data-science-based rules, risk-based detection, and leveraging as much entity-based correlation as possible.

TTPI’s: Extending the Classic Model by Andrew VanVleet

Tactics, Techniques & Procedures (TTPs) is a table-stakes term in our industry. It binds our understanding of attacker behavior into a common lexicon. Within this lexicon, MITRE ATT&CK reigns supreme, and they have some generally agreed-upon definitions within their ATT&CK FAQ. Basically, in order to understand MITRE ATT&CK, you have to understand their nomenclature of TTPs, where:

• Tactics describe an adversarial objective, such as initial access

• Techniques describe how an attacker can execute some operation to achieve that objective

• Procedures describe the implementation details of a technique in a given environment

In this post, VanVleet challenges this model because the specific details of how an attack is carried out at the Procedure level can sometimes be vague. I think this is by design on MITRE’s part, because the procedure to achieve it can differ depending on the environmental context I mentioned earlier. He makes the analogy that Procedures are like a cake, not necessarily a recipe. He proposes the concept of Instance, which is the recipe itself, to achieve that procedure.

ATT&CK does get close to this via Detection Strategies. As an example, VanVleet looks at T1070.001, Indicator Removal: Clear Windows Event Logs. The MITRE page includes a description of how this can be achieved, but it seems high-level enough that some more detail on the recipe would be helpful. The detection strategy can provide more clues from an event-ID perspective, but without the technical implementation, it may be hard to recreate and test. Here’s his idea of what an Instance section could look like:

This could be helpful for detection engineers who want to recreate the attack in their own environment to test their telemetry generation and detection rules.

I’ve always had a hard time with the Pyramid of Pain for this exact reason. The “TTPs” part at the top of the Pyramid can encapsulate so much work, without any ability to reverse-engineer how the attack is captured. In fact, I’ve always thought TTPs/Tools should be combined, because almost every Procedure contains some level of tooling to capture the attack.

In the spirit of alliteration, and perhaps more as a thought exercise, he proposes the “Pyramid of Permanence”.

Basically, Procedures are what we want to capture, and everything below the tip of the Pyramid are Instances that supports the procedure. It’s an interesting thought experiment, and as long as it serves as a lexicon to drive the conversation on better modeling, I’m all for it.

Centralized Suppression Management for Detections Using Macros & Lookups by Harrison Pomeroy

Detection rule efficacy is the practice of curating rule sets that balance precision, recall, and the cost of triage. New detection engineers typically think about rules being the only place you can apply logic to help manage this balance. A more precise query that accounts for benign behaviors, given the tactic or technique, can increase the likelihood of capturing true positives. But there are other capabilities in SIEM technologies and software engineering practices that can perform filtering and suppress alerts in more dynamic, context-aware ways that align with the threat landscape or your environment.

This post by Harrison Pomeroy details the power of Splunk’s macro and lookup table functionality to perform suppression of alerts without re-deploying rules. A suppression is a concept in which detection engineers deploy a capability to dynamically mute alerts, thereby reducing the cost of both false-positive generation and the subsequent need to tune a rule on small fields. It also makes the rule more resilient because it can account for external factors related to benign behaviors, such as known service accounts, scheduled tasks, or internal tooling.

Harrison leverages Splunk’s macro and lookup table features to achieve this.

The above Mermaid diagram shows his really clever setup. When you apply macros to each of your Splunk rules, you can start bringing in logic to evaluate whether suppressions are enabled for the rule (the T value), and then specify a lookup table to find additional alert logic to append to your original rule to suppress false positives.

The above example suppresses alerting on any user called svc_backup. The macro executes based on the T value and performs a lookup in a table relevant to the PShell Alert rule. svc_backup is in the table and uses a NOT() filter to prevent an alert if svc_backup is present. The suppressed green box ensures the alert doesn’t fire, and the Alert red box fires because the user is jsmith.

This type of suppression occurs at query time, before the alert is generated. There are other suppressions you can apply before a log hits the index, or after the alert fires. This is a great topic for my Field Manual series, so thank you, Harrison, for the inspiration!

👉️ Read Online if Clipped 👈️

The Detection Engineering Baseline: Hypothesis and Structure (Part 1) by Brandon Lyons

Baselining is an overused term in this field because, at least in my experience, it’s a hand-wavy marketing term. You’ll read about a product that’ll perform baselines of your behavior and environment, and it’ll alert you if it detects something abnormal or outside that baseline. In practice, this works, but the opaqueness of some of these methods makes it hard to understand how it happens.

This is why posts like Lyons help cut through the opaqueness and show the receipts of how to do this in practice. And to be honest, it’s nothing groundbreaking, only in the sense that the concepts Lyons proposes here are part of entry-level statistics literacy. Which is why I’m pretty opinionated on the engineer of detection engineer. Don’t get it twisted: although the concepts in this post are entry-level statistics, understanding the application requires deep security expertise.

Lyons lays out a 7-step, repeatable process to establish a detection baseline, quoted here:

• Backtesting of rule logic: Validate your detection against historical data before deploying

• Codified thought process: Document why you chose specific thresholds and methods

• Historical context: Capture what your environment looked like when the baseline was created

• Reproducible process: Enable re-running when tuning or validating detection logic

• Foundation for the ADS: Feed directly into your Alerting Detection Strategy documentation

• Cross-team collaboration fuel: Surface insecure patterns and workflows with data-backed evidence

• Threat hunting runway: When alert precision isn’t achievable, convert the baseline into a scheduled hunt

This process succinctly captures a well-thought-out detection process. Without data, how can anyone possibly deploy detections that will fire? Without context around that data, how can anyone possibly believe the rules that are firing outside of the baseline?

They step through the 7 steps here using a CloudTrail API example. Basically, Lyons tries to map out what anomalous behavior looks like for CloudTrail access across an environment. The statistics section focuses on a modified Z-Score. Here’s the rundown:

Security metrics (API calls per day, login attempts per hour, file accesses) approximate a normal distribution (a bell curve), especially when aggregated over time. This means that:

• Most values cluster around the median (middle value)

• Extreme values become increasingly rare as you move away from the center

• The distribution is symmetric

To establish a baseline, Lyons collects historical data, such as 30 days of activity, and computes two key statistics:

• Median - the middle value

• MAD (Median Absolute Deviation) - measures spread around the median

When a new value enters your queue, you compute the Modified Z-score, which is the distance-via-standard-deviation of that value from the median. Modified Z-score is really good at capturing outliers, versus the regular Z-score, which focuses on standard deviations from the mean, and can be sensitive to outliers.

An outlier can be, according to Lyons, creating administrative credentials at 3am to an abnormal amount of S3 bucket accesses, perhaps used for exfiltration. Here’s a graphic I prompted Claude to create to drive this point home:

This type of rigor removes the guessing game about whether events are absolute measurements. Is 1000 API calls weird, or is 100? Is 10 pm an acceptable window for Administrator access, or is 5 pm? By looking at the standard deviations away from the median, you focus on relative measurement. It removes the human judgment about the absolute weirdness of an event, and whenever you remove a human from a large data problem, you get a bit closer to sanity.

Lyons created a follow-along Jupyter notebook with synthetic data to recreate the measurements in his blog.

Welcome to Issue #154 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I’m back from Spring Break and happy to report I have no sunburns. New England Spring is here as well, and it feels like the Northeast U.S. is coming out of hibernation

• I just booked Hacker Summercamp (BlackHat & DEFCON), so excited to see folks there. If anyone wants to meet up/host an event/drink Miami Vice by the pool, HMU

  • For my BJJ fam: if anyone wants to train at Jeremiah Grossman’s Smackdown or hit an open mat during that week, let me know :D

Webinar with Forrester: AI x Security Operations, What Works and Doesn’t Work

I’m hosting a webinar with Allie Mellen from Forrester tomorrow, where we’ll be diving deep into security operations and how AI is working and not working for all of us.

We’ve had awesome discussions around this in the past. Feel free to register and come roast me in the chat!

Register & Roast Zack

💎 Detection Engineering Gem 💎

A quick look at Mythos run on Firefox: too much hype? by Antide Petit

The talk of the town last week was Mozilla's blog post on how they used Anthropic’s mysterious and powerful Mythos model to find and fix 271 vulnerabilities. The blog post itself isn’t bragadocious in the way you might see vulnerability reports; in fact, it has a level-headed take on how the Mozilla team is hopeful about the scale LLMs can find vulnerabilities, but that no singular vulnerability found was something that a human couldn’t find:

So far we’ve found no category or complexity of vulnerability that humans can find that this model can’t. - Petit

They used the term “vertigo” to describe how jarring the capabilities of LLMs are in changing our perception of defense.

Luckily, this post by Petit helps ground the announcements even further into reality, with some objections to the hype of the news. Petit reviewed Firefox’s issue trackers and commit history to gather more details about the 271 reported bugs. Petit went through the commit history to map bugs to CVEs, classify by exploitability and attack surface, and figure out which of the 271 actually met the bar for a CVE or had a PoC.”

They vibe coded an excellent visualization tool with their findings, located here, with a neat dashboard shown below:

The 12 CVEs shown here tell a different story from the 271 vulnerabilities. Granted, the Firefox team did not say they issued 271 CVEs, but it depends on how we interpret those vulnerabilities and whether they are presented as exploitable or meet the bar for a CVE. The other finding here is that a vulnerability patched for a defender has a tighter distribution of usefulness than one found that is useful for offensive security purposes. A fully exploitable vulnerability still won’t guarantee a Firefox browser breakout, and you typically see these chained together to fully break out of the sandbox.

Petit ends the blog with a section on defender and attack relevance that captures my last point much better than I could ever explain it. Foundation models are proving themselves to be a useful tool for increasing the velocity of defense at a scale that sometimes gives us vertigo. But as an offensive security tool, it may not seem as useful or exciting because of the complexity of building a fully exploitable chain against an extremely hardened piece of software like a web browser.

The operational details of the research matter - Petit

Until this restraint on the opacity of research details becomes more transparent, it’s hard to separate the wheat from the chaff among blog post announcements from foundational labs. So, remain hopeful, but the hype is deliberate to build buzz, even though Anthropic does a good job of balancing this hype so it doesn’t seem disingenuous.

🔬 State of the Art

Streaming Logs to RSigma for Real-Time Detection by Mostafa Moradian

I covered Moradian’s RSigma tool in a previous gem, and he has been busy since then :). RSigma is a Rust binary that evaluates Sigma rules against JSON logs without a SIEM. Since that post, three releases have added some neat new features: NATS and HTTP inputs, a hot-reload feature for rules, observability via Prometheus metrics, and persistent correlation windows backed by SQLite.

Moradian walks through a well-known Okta cross-tenant impersonation scenario to show how these new features work in practice. The four SigmaHQ rules covering that attack (proxy login, MFA deactivation, privilege grant, IdP creation) each fire independently on events that are individually defensible.

The temporal_ordered correlation rule ties them together, requiring all four to fire in sequence from the same actor.alternateId within 30 minutes. Without stateful windowing across events, you risk creating noise on these four rules that may not be correlated. The field-mapping pipeline that reconciles Sigma rule field names with Okta’s camelCase API schema is what makes the whole thing portable. Moradian frames this as one of the hardest parts of detection portability. Vendors certainly take this for granted and leave the work to detection engineers, but Sigma is the closest to standardizing this.

RSigma is not a SIEM, as Moradian puts it, but it’s an impressive feat to build a self-contained Rust binary that operates much like one. For teams doing pre-SIEM rule validation or forensics, it’s a solid plug-and-play option for certain scenarios. It’s also a great read for understanding the deeper architectural challenges software engineers face when building high-volume distributed detection systems.

Are Detection-as-Code Pipelines Overrated? by Harrison Pomeroy

Detection-as-code (DaC) has been the gold standard maturity milestone for security teams for years. The goal of DaC is straightforward: provide governance, guardrails, human review, and validation of detections before they ever touch a production instance. It attempts to minimize regressions, detection drift, and cost increases through the lens of CI/CD and widely used SRE concepts.

Much like everything in security, agentic workflows present opportunities to improve this architecture or remove it altogether. So, in this post, Pomeroy explores this topic with an honest look at how we can scope out several portions of a DaC pipeline and move the work toward the agent running on the detection engineer’s laptop. Schema validation, metadata creation and documentation, linting, and accuracy validation agents for backtesting and accuracy checks can mostly be handled by an agent before it ever hits a pipeline.

We had many of these tasks within CI/CD because we expected humans to make errors. The governance aspect of DaC is attractive because centralizing knowledge around schemas and pre-deployment checks is deterministic by design. As Pomeroy points out, we perhaps overcorrected regarding the necessity of deterministic checks for safety, and an agent can provide both safety and speed. The DaC pipeline still exists, but in a much leaner form that still requires humans for approval.

Detection Pipeline Metrics by Scott Plastine

This short-but-sweet post on detection metrics is a continuation of Plastine’s post on Detection Visibility Metrics. I highly recommend reading the Visibility Metrics post, from which I learned two insights:

• Visibility is just as important as detection itself. There is no rule without telemetry, and you should treat log sources as an asset as much as you treat developer laptops

• We focus too heavily on rule metrics, such as coverage, and neglect business-level metrics like the number of users, coding environments, and servers we protect

After visibility, according to Plastine, you should focus on metrics within your logging pipelines themselves. I love how he used the Funnel of Fidelity as the inspiration for some of these measurements. If we don’t want to “clog the funnel”, we should look to reduce the amount of noise that arrives at alert inboxes. You reduce the amount of noise that makes it to alert queues by building more robust rule sets, risk scoring through composite rules or risk-based alerting, and building pipeline features that flatten or aggregate telemetry rather than sending in a ton of logs at once.

Midnight thinking on browser extension security by Anya Nessi

This is a great late-night musing piece on how it’s going to be harder to differentiate code authors as a detection signal due to agentic coding. The anchor is Red Canary’s Cyberhaven incident analysis, where the compromised extension update scored a modified z-score of 75.38 against the extension’s historical entropy baseline. For context, 3.5 is already a strong statistical outlier. A score of 75 means the injected script’s entropy was so far outside the distribution of the legitimate codebase that attribution to the same author was statistically implausible. I covered the modified z-score in Issue 145 if you want more background.

Nessi built her own entropy-based detection pipeline along similar lines, and it works. The question she’s grappling with is what will happen to this technique as LLM-assisted development becomes the norm for both legitimate developers and attackers. If both parties are writing code using tools trained on overlapping data, the distinct human authorship fingerprints that enable entropy-based detection begin to drift.

☣️ Threat Landscape

📦🔗 TeamPCP News

TeamPCP was back in the news this last week! These attacks don’t seem as impactful as the several I covered earlier this month, but there are some worthwhile callouts about updates to TTPs.

Malicious Checkmarx Artifacts Found in Official KICS Docker Repository and Code Extensions by Socket Research Team

The group compromised multiple Checkmarx distribution channels simultaneously: the official checkmarx/kics Docker Hub repository had trusted tags overwritten with a trojanized KICS binary that exfiltrated secrets during infrastructure-as-code scan runs for Terraform, CloudFormation, and K8S configs. Checkmarx ast-vscode-extension had an orphaned 2022 commit injected carrying a payload that runs via Bun and exfiltrates secrets, including MCP config files. It looks like the Bitwarden CLI npm hijack was part of the same campaign, and I wrote about this below.

TeamPCP Campaign Spreads to npm via a Hijacked Bitwarden CLI by Meitar Palas

In the next part of the campaign, the group compromised the npm CLI of the well-known password manager BitWarden. According to JFrog research, the group hijacked @bitwarden/cli version 2026.4.0, keeping the legitimate Bitwarden branding intact while rewiring the installation scripts to download Bun and execute a payload that attempts to exfiltrate GitHub tokens, SSH keys, and AWS/GCP/Azure secrets, as well as GitHub Actions secrets. The interesting part here, which I haven’t seen before, is that the malware explicitly targets ~/.claude.json and MCP config files, potentially marking a shift to use secrets from coding agents to pivot into victim environments.

Other News

I Left Port 22 Open on the Internet for 54 Days. Here's Who Showed Up. by Arman Hossain

This was a fun honeypot research project write-up in which Hossain deployed a basic SSH honeypot on a cheap VPS to collect and analyze connection attempts and attacks. Nothing here seems out of the ordinary or new from the sense of novel attacks, but it goes to show how noisy the Internet is and how easy it is to be targeted by Internet-wide scanners. Attackers attempted to run default credentials for well-known IoT devices, tried to download binaries to have their servers join a botnet, and had some level of hands-on keyboard operators interacting with the server. This would be a great experiment and exercise for folks getting into threat research and log analysis to build a server like this and analyze the logs.

Fibergrid: Inside the Bulletproof Host for 16,000+ Active Fake Shops by Harry Freeborough

Bulletproof Hosts are organizations that provide IP leasing space for customers and are known for not responding to takedown requests from abuse reports and, often, to law enforcement preservation requests. They are impressive feats of misdirection in that these organizations tend to layer themselves through shell companies and hard-to-contact administrators to maintain anonymity.

Fibergrid is a particularly unique bulletproof hoster because of its origin story. Netcraft Research has been tracking Fibergrid and attributed 16,700+ active fake shops and an IP address pool that traces back to the Great African IP Address Heist. Netcraft found that the servers are actually sitting in Equinix facilities across the US, UK, and the Netherlands, not Africa, which Netcraft argues gives Western law enforcement a real pressure point.

🔗 Open Source

F2u0a0d3/goodboy-framework

I was hoping to get a dog picture or instructions for training a dog to be a good boy in the README. Instead, I found an excellent resource for people trying to learn malware development, analysis, and detection engineering on Windows using Rust. There are 15 lessons or “stages”, and each one has a particular technique it’s trying to teach you to learn. They integrate malware technique development, such as direct or indirect syscalls, with analysis techniques for finding what you wrote along the way.

weirdmachine64/SharkMCP

Locally-run MCP server that provides tooling for local agents to perform PCAP analysis using Wireshark’s sharkd API. There are close to 20 tools that weirdmachine64 exposes for clients, and so you’ll want to add this one to your CTF arsenal, especially if you are looking at pcap files.

trailofbits/trailmark

Trailmark is a tool for visualizing code paths and dependencies. You feed it a codebase to analyze, and it’ll construct an abstract syntax tree in Treesitter format and pass it to a graphing function. You can then query the graph for specific classes or code paths, as well as use their querying capabilities to perform reachability analysis, annotate functions, find dependencies, or look for “paths in between” two nodes.

synacktiv/pike-agent

Pike-agent is an LLM assistant that reads strace telemetry and performs analysis based on the prompts you give it. For example, if a binary crashes every time you run it, you can feed it to pike-agent, and it’ll help you debug the root cause. I think the cool use case here, and I might be biased in security, is the malware analysis functionality :).

anondotli/awesome-privacy-tools

Yet another awesome-* list, this time focused on privacy tools. I’m surprised something like this hasn’t been made yet, but it’s nice to see an aggregation of useful tools that can help improve your OPSEC. Might be especially useful if you are a threat researcher or intel specialist doing cybercriminal research on underground forums.

Welcome to Issue #153 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I’m spending time in the Caribbean this week with my family and wish I could write this newsletter everyday with nice weather and an ocean nearby. Luckily New England does have the Atlantic, but it’s not the same for two reasons. First, the fruit is so much tastier down here. Lastly, I can’t find an Oscar the Grouch with dance moves quite like this Oscar :D

  

• I’m hosting a webinar with Allie Mellen from Forrester on April 30th where we’ll be diving deep on security operations and how AI is working and not working for all of us. We’ve had awesome discussions around this in the past. Feel free to register and come roast me in the chat

  Register & Roast Zack

Sponsor: Push Security

Register for a brand new research-focused webinar series from Push Security

The browser is the place where modern breaches happen, powered by a huge amount of attacker innovation — countless ClickFix variants, new malvertised phishing campaigns, and device code phishing attacks being powered by brand new PhaaS kits and AI tools. And we’re only in April.

Join Push Security threat researchers, along with incredible guests like John Hammond, Troy Hunt, and Matt Johansen, in a brand new webinar series deep-diving into the State of Browser Attacks.

Register Now

💎 Detection Engineering Gem 💎

Internet Protocol Version 8 (IPv8) by Jamie Thain

Ok, here’s a confession, but it’s more of a brag than something to be embarrassed about. When I went to college, I studied networking and network security extensively, as the 2008-2012 era had a strong focus on it. A lot of what my classes entailed was setting up large VM farms and networking devices on server racks. I lucked out because I got really good at networking, and part of that involved studying RFCs. I love RFCs. My favorite RFC is 1034, DNS, the coolest protocol in the modern Internet.

My co-worker sent out a link this week amid the Opus-4.7 news about the IETF skipping odd-numbered Internet Protocol RFCs and moving directly to IPv8. This specific proposal (not yet accepted) attempts to address the fragmentation of IPv4 & IPV6 networks around IP address assignment, DNS, NTP, telemetry, authentication, route validation, and access controls.

Jamie Thain, the proposal’s author, suggests that these fragmented services can be reconciled through a singular concept called a Zone Server. This Zone Server assigns DHCP leases that contain network information for everything I listed above. As Thain puts it:

The most interesting part of this proposal, IMHO, is the authorization model. Every manageable element in an IPv8 network is authorized via OAuth2 JWT tokens. Like what Thain said in the quote, when a new device joins, it sends a single DHCP8 Discover and receives a lease containing the device’s configuration, including a JWT. The “OAuth2 Authority” can be hosted on a home router or on an external IdP such as Google Workspace, Okta, or Azure AD. The Zone Server stores the public keys for these authorizations, so local devices can verify token validity with the Zone Server before interaction. It reminds me a lot of Tailscale, but unlike Tailnets, rogue devices can still join your network; you just need to make sure peer devices validate tokens before they talk to it.

The second most interesting thing is DNS. Every outbound connection must have a valid DNS8 lookup before it traverses the broadcast domain; otherwise, it is automatically blocked. This helps protect against malware infections calling back to a C2 server.

As Claude pointed out, this authorization layer exists at a higher layer in the OSI stack. Attacks can still occur on lower layers. Putting my networking nerddom aside, it’s refreshing seeing how Internet-scale engineers are thinking about solving issues around security and having solutions being a feature of the protocol itself. This prevents service fragmentation, poor visibility, and the need to stitch together different security layers and hope they work.

🔬 State of the Art

This was a fun episode of Discarded where the hosts interviewed their coworker, Stuart Del Caliz, who’s a threat detection engineer at Proofpoint. When I think of writing rules and detections, my mind usually drifts towards researching and deploying log-based detections in a SIEM. Del Caliz, on the other hand, focuses on Suricata & YARA rules for malware using appliance products via the Emerging Threats Ruleset. The team has a robust malware sandbox that enables Del Caliz and the detection team to identify patterns in C2 traffic and binaries and to generate alerting and blocking mechanisms for customers.

Luckily, the Emerging Threats Ruleset is available for download, so you can set up your own Suricata lab to test it out.

macOS Threat Investigation Benchmark by Cotool Research

Friends of the newsletter Cotool published their latest research benchmark on the efficacy of the latest foundational model around macOS investigations. What makes this interesting is that, unlike CTF events, which have a clear incremental path to success, this benchmark uses logs and telemetry from an Odyssey stealer infection. The agents were given access to 14 log sources across hundreds of thousands of events and had question harnesses across 36 tasks in incident response, threat hunting, and detection engineering.

GPT-* models performed very well across Accuracy, Speed, and Reliability, and were middle-of-the-pack in cost. I think the most interesting finding here is that Cotool rewarded models based on task difficulty. For example, Incident Response was the hardest and most expensive among the tracks, and GPT-5.4 and 5.3 Codex had the best combination of accuracy and cost efficiency. Opus 4.6 had the same accuracy but cost nearly double that of GPT-5.4

Detecting CI/CD Supply Chain Attacks with Canary Credentials by Alessandro Brucato

The Tracebit team just released a clever canary detection mechanism for supply-chain attacks against GitHub Actions, similar to what we saw with the TeamPCP campaign a few weeks ago. Security teams can pull in their community edition GitHub action, which generates per-run canary credentials, such as AWS tokens. If your security controls fail or you fail to detect an attack, you can use these tokens as a reliable alerting tool that points to the repo and specific GitHub action.

They have a community edition, which is always great to see, because you can sign up and try it without all the vendor marketing gates and FUD that is commonly shoved down security people’s throats :).

Codex Security by OpenAI

The foundational labs are leaning more and more into cybersecurity use cases. With Mythos’ release last week, OpenAI released a Codex Security preview that allows vetted cybersecurity professionals to use an advanced version of Codex to scan their repositories for vulnerabilities. The “vetted” part of this announcement is interesting because Mythos was heavily gated in their announcement, whereas OpenAI has made it easier for folks to apply and get access if they meet certain verification criteria.

Sponsor: Spectrum Security

Stop the Grind and Kill the Detection Backlog with Spectrum

You know the grind: Research the gap. Understand the environment. Write the logic. Tune it. Deploy it. Watch it break when something upstream changes. Repeat. Backlog never shrinks.

Spectrum is now available. One platform that continuously maps your coverage, authors deployment-ready detections tailored to your stack, and keeps them resilient, so your expertise drives strategy, not maintenance.

See It In Action

☣️ Threat Landscape

Two U.S. Nationals Sentenced for Facilitating Fraudulent Remote Information Technology Worker Scheme that Generated $5M in Revenue for the Democratic People’s Republic of Korea by U.S. Department of Justice

Why is it always someone from New Jersey?

Kidding, unless you are a Jets fan. The DoJ released a sentence announcement for two U.S. nationals who became facilitators for Wagemole over several years. According to the announcement, they helped generate over $5 million USD of revenue for DPRK, stole identities of close to 100 U.S. persons, and worked at 100s of U.S. companies.

I post a lot about the DPRK in this newsletter because it hits so close to home, given that I work in tech. I also post a lot about DPRK in this newsletter because I have to follow up with these Wagemole stories by watching this Key & Peele sketch:

NIST Updates NVD Operations to Address Record CVE Growth by NIST

NIST released updated guidelines on how they will perform CVE enrichment moving forward. They’ve had nearly two years of turmoil trying to keep up with enriching CVEs for their National Vulnerability Database. I think this failure to enrich every CVE is due to two things: funding shortages and AI. I’ve linked a ton of stories here from open-source maintainers like Daniel Stenberg, saying that AI is creating too much vulnerability research slop. But I do believe recent CVEs have been at least AI-assisted, which helps increase velocity.

In their announcement, NIST says it will prioritize CVEs into three categories: CISA KEV entries, vulnerabilities in software used by the federal government, and CVEs for Critical Software, as detailed in an executive order from 2021.

There are other playbook changes regarding severity scores and modified CVEs, but IMHO, the significant reduction in CVE enrichment is the big news here. I hope we can find a way, as a community, to fund efforts to enrich CVEs in lieu of the NVD.

MCP Supply Chain Advisory: RCE Vulnerabilities Across the AI Ecosystem by Moshe Siman Tov Bustan, Mustafa Naamnih & Nir Zadok

The Ox Security Research Team found four attack surfaces in Anthropic’s MCP Protocol implementation. These attack surfaces led the research team to disclose vulnerabilities to dozens of open-source repositories and get to 10 CVEs and counting. The way it works is how this software sets up MCP servers and what they allow as input into their configurations. The command injection is via exposed tools that can add an MCP configuration. When you add an stdio transport, you can specify an arbitrary bash command that allows remote code execution.

The team disclosed this vulnerability to Anthropic but the protocol specifically allows this option and is by design. The problem here is when users of these MCP-enabled projects don’t know about the insecure configuration and deploy it to the Internet allowing the remote code execution. There are situations where this RCE can be unauthenticated or authenticated, but for the most part, it points out that anytime you allow arbitrary configurations to be uploaded by users, you risk exposing yourselves to attacks like this.

Understanding security warnings when opening Remote Desktop (RDP) files by Microsoft

Microsoft finally released security updates to help warn users of malicious RDP files before they become victims of an infection or social engineering attack. Attackers leverage RDP files to initiate remote connections to victim computers, often using them to steal files, take pictures or steal contents of your clipboard. Now, users of the latest security update will get warnings when double-clicking on these files, hopefully preventing some of these infections from happening.

With the advent of ClickFix, my hopes aren’t too high, but sometimes security is about incremental steps versus massive swings.

🔗 Open Source

google/magika

Magika is file on steroids. It uses a deep-learning model under the hood that helps classify files with what they claim is 99% accuracy. It was trained on 100 million+ samples and 200+ content types. The cool part here is that this is the model and tool used by Google to help detect filetypes on Gmail, Drive and Safe Browsing. I imagine they can use this to route files based on their content types to different internal security services for scanning.

mukul975/cve-mcp-server

Locally ran MCP server that helps researchers and defenders connect to 27 (!) security tools for CVE lookup and enrichment. Unfortunately you won’t get much data anymore from NVD, but it has some great integrations with ATT&CK, internet scanners and even VirusTotal.

Hainrixz/cyber-neo

Cyber Neo is a Vulnerability research plugin for Claude Code. It has 11 security domains it tries to scan for, and each domain has toolsets and markdown instructions for Claude to execute to render findings.

tahaafarooq/Fenrir

GoLang credential and secrets harvesting tool that uses eBPF to skim credentials off from syscall events. It’s like a Linux rootkit that only cares about secrets. It has some interesting capabilities, such as memory-only execution and anti-detection capabilities. It tries to intercept secrets across SSH, PAM, the command line and does some file-based discovery for things like API keys and cloud secrets.

jsmonhq/xnew

xnew is a low-footprint and fast file appender. It is contextually aware of every line inside the file, and it will only append unique lines not already present. This has always been a pain for me everytime I’ve had to cat x | uniq | sort | uniq , which can sometimes take forever when the file is super large.

Welcome to Issue #152 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• The sun is staying out later and coming up earlier. There’s nothing better to me than an early morning sunrise :)

• I finished my book about the Marquis de Lafayette, Between Two Worlds, and it was fantastic. I’m already reading a new one about the ugly truths of living on Mars called A City On Mars. A former NASA Chief Economist recommended it on a podcast

• I’m excited for an upcoming beach vacation in the Caribbean for some much-needed sun and relaxation. I’ll still be putting an issue out, so there won’t be a gap in coverage (ha)

Sponsor: Adaptive Security

Can Your Team Spot an AI Deepfake Attack?

Today's phishing attacks involve AI voices, videos, and deepfakes of company executives.

Adaptive Security is the first security awareness platform built to stop AI-powered social engineering.

Adaptive protects your team with:

AI-driven risk scoring that reveals what attackers can learn from public data
Deepfake attack simulations featuring your own executives
Interactive, customizable training content

Tour The Platform (3 minutes)

💎 Detection Engineering Gem 💎

Measuring What We’re Missing by George Chen

In this post, Chen gives readers some honest thoughts and super reasonable metrics around measuring detection efficacy. We tend to fall into the true-positive/false-positive trap because they are the easiest to measure and explain. False negatives are the most risky, but if you only rely on a security incident where an alert failed to fire, they can really affect your detection engineering operations, because you can only measure when things go wrong. Your operational work should revolve around identifying coverage gaps (false negatives) and eliminating unnecessary work (false positives).

These metrics can fall into “busy work”, when you really want to show impact. You also risk making your coverage gaps harm your operational score instead of celebrating them.

Chen’s fix is to separate detection efficacy into two signals:

• An effectiveness score (how well do tested controls perform?)

• A discovery count (how many new gaps did you find outside of testing?)

The “under test conditions” qualifier is the important part. This isn’t a coverage number. It’s a performance number scoped to what you’ve actually challenged through red teams, purple teams, BAS, and threat hunts. If 50 techniques are executed and 10 are missed, you now have a denominator, a defined scope, and a measurable gap. Without that structure, a miss is just an observation.

The discovery count stays separate on purpose. If you lump newly found gaps into the denominator, the more unknowns you surface, the worse your score looks. That creates a perverse incentive where teams stop looking for blind spots because finding them risks tanking the metric. Chen’s answer is simple: keep it as a standalone count. “3 new gaps discovered and addressed in Identity & Access this quarter.” Effectiveness tells you how well tested controls perform. Discovery tells you how much you’re still missing.

I’m seeing metrics like this more often in security operations, where we’re starting to describe the health of the system, similar to what Site Reliability Engineering departments do. Chasing 100% accuracy is meaningless due to the Precision and Recall Problem, but showing any kinks in the armor can come across as unpreparedness. Owning the idea that you need to curate and maintain a ruleset, just like you maintain a cloud or on-prem environment, is a more stable approach for your sanity and for business outcomes.

🔬 State of the Art

Hunting Malicious Teams Delivered Links via Endpoint & Cloud Telemetry Correlation by CipherSecy

This comprehensive threat hunting report highlights a rare but effective attack scenario around Microsoft Teams. In any modern workspace chat application, you can talk with your coworkers and external people like contractors, vendors, or customers. So, something like Teams or Slack can serve as an excellent pivot point for threat actors, since they gain direct access to your DMs, and the telemetry isn’t as well-documented as with phishing emails.

CipherSecy built the following hypothesis before their hunt:

A compromised third-party account sends a malicious link via Microsoft Teams with the intent of compromising an internal user’s identity.

What follows are their findings and documentation on available telemetry to help catch these types of attacks via Teams. A hunt like this uncovers a ton of nuances and peculiarities in the attack flow from a visibility perspective:

Teams launches an in-app browser via a CLI command. The browser can link to malicious downloads or phishing sites, so making sure you have an EDR that can provide that telemetry is important. The cool part here IMHO is the rich context from within the CLI command:

C:\Program Files (x86) \Microsoft\Edge\Application\msedge.exe"
--single-argument microsoft-edge:///?url=https://github.com/notsosafelink&
source=teams&treatment=4445&form=MY02BU&qpc=955403648535
&oid=<RCV-OBJ-ID>&hubappid=bc25fcef-8964-4e72-8287-23e2b496c128
&hubappsubpath=embed-client/chats/19:<SNDR-OBJ-ID>_<RCV-OBJ-ID>@unq.gbl.spaces
/view&hubappparams=hostCtx=edge&layout=singlePane&src=teamsLink
&messageId=<MSG-ID>&oid=<USER-OBJ-ID>&loginHint=<RCV-UPN>
&startTimeStamp=1773993512074&correlationId=<GUID>

CipherSecy points out two things here. One, —-single-argument indicates a process spawned Edge programmatically, which helps reduce the noise of manual browsing. Secondly, src=teamsLink means it was spawned from Teams itself. Both turn into high-value signals, and throughout the rest of the post, they show some of their KQL queries to perform additional hunting and inspire some detection opportunities.

Joshua Saxe

Exploits don't cause cyberattacks

As far as I can tell, here’s the median model of attacker behavior people are using in their Claude Mythos hot takes…

Read more

5 months ago · 10 likes · 5 comments · Joshua Saxe

Mythos has been the talk of the town since its preview release on April 7. The industry reacted to the hype with mixed reactions. On the hype side, it’s an extremely impressive model and deserves its accolades for vulnerability research and exploitation. In fact, Anthropic is worried enough about the model that it created an invite-only program, dubbed Glasswing, to give early access to companies that will initially use it to find and fix vulnerabilities.

On the other hand, the incentive structure of frontier labs like Anthropic is to build hype and generate buzz. And when you generate buzz around the security industry, you will get pushback against the hype, whether you want it or not. I believe Saxe’s pushback in this article has the best-grounded arguments to help us brace for impact without burning too much energy bracing too hard.

Mythos, much like Opus’ release, will fundamentally change a lot of our capabilities. But much like Opus, our security capacity is bounded by more than just computation and prompting. Saxe frames this argument with a thought experiment. If these frontier models changed the game for synthetic voice and text, have we meaningfully seen an explosion of activity in social engineering and phishing attacks? The key here is “explosion”, because that’s what it seems like the Mythos release is warning the industry about, but instead of phishing, it’s vulnerabilities.

I do wish there were some investigation from Anthropic on the detection and response front. Mythos will clearly help the vulnerability side of the house, but what about deep investigations, rule writing, or threat hunting? Frontier models have fundamentally changed blue team operations in these fronts, but I don’t think it’s ruined the status quo. We’ve certainly become better prompt engineers, though :).

Myth & Mythos: Where Do We Go From Here? by Joe Slowik

It’s pretty apt that Joe Slowik wrote a blog about Mythos on his blog named “Stranded on Pylos”. I really enjoyed reading this essay, mostly because it highlighted some of the intentional or unintentional decisions Anthropic made when announcing Project Glasswing. Specifically, the lack of non-American companies and the focus on tech & IT rather than critical infrastructure or healthcare organizations.

Joe is a staunch advocate for critical infrastructure security research, especially around OT systems. He offered a critical but fair take on the initial release of Project Glasswing, lacking any focus on these areas. In all fairness, as he points out, many of these large tech companies do build and maintain products for critical infrastructure networks, but there isn’t enough information from Anthropic to confirm whether they are considering the threat model for these networks.

Admittedly, I think it’s a Catch-22. If Anthropic brought in a Siemens, and maybe didn’t bring in Apple, would we be making the same argument? Probably. And the marketing is well done, capturing the attention of major news outlets worldwide. Though Anthropic, in my opinion, has done the most to demonstrate its commitment to AI safety research, I feel like they are more trustworthy for the time being, especially when they say something is “too dangerous right now.”

Webex-ploitation by Grumpy Goose Labs

I first featured Grumpy Goose Labs in Issue 11 (!!), and since then, they’ve done a ton of research on hunting for Fake IT Workers. In Issue 138, I wrote an analysis of their fantastic research on hunting for KVM Switches in Crowdstrike, which can be a great signal for facilitators who gain fraudulent employment. In this post, they switch their hunting methods to look for Webex sessions used by facilitators in a similar way.

I find it insane how RMM software, like Webex, has poor audit logs, logs everything locally, and provides opaque logs that make it a lot harder to detect and hunt for this activity. I ran a cursory search on GitHub for any log-shipping pipelines that parse, normalize, and ship these logs to providers, and I didn’t find any.

The craziest find in this research is how WebEx has keylogging capabilities. It’ll record keyboard firing events to the local log files, and so theoretically you can: a) spy on your employees, b) run malware that ships these logs off to a C2 for password collection, or c) hunt for TTPs by some of these IT Workers.

☣️ Threat Landscape

Our response to the Axios developer tool compromise by OpenAI Security

The OpenAI Security team published a security update on the impact of the Axios supply-chain compromise on their macOS signing process. According to their security team, the GitHub action that signs the binaries for their macOS apps, such as Codex CLI and ChatGPT desktop, was compromised and downloaded the malicious Axiox 1.14.1 version.

Based on research published over the last week and a half, many of these compromised builds failed due to peculiarities in their code, but OpenAI revoked and rotated the signing certificate out of an abundance of caution.

Tracking Adversaries: EvilCorp, the RansomHub affiliate by Will ‘BushidoToken’ Thomas

Following my Threat Landscape coverage from last week’s issue, threat research G.O.A.T. BushidoToken’s timely issue on EvilCorp helps tie their operations under the newer and active RansomHub affiliate program. Since the U.S. sanctioned EvilCorp, it has become much harder for victims to pay the group after they suffer a ransomware attack. This leads groups to rebrand as new groups or join affiliate programs to continue their operations, removing a significant financial hurdle to their success.

Will’s survey of infections from the last few years of ransomware attacks helps tie them to EvilCorp because of the use of the SocGolish malware. This is about as close as you can get to attribution with only pure technical data without relying on HUMINT, such as law enforcement or doxxing.

Inside an AI‑enabled device code phishing campaign by Microsoft Defender Research Team

This post by Microsoft Defender Research highlights a phishing operation tracked under the EvilTokens phishing group. It focuses on DeviceCode phishing, where a threat actor abuses an authentication flow primarily used to sign in to Microsoft accounts associated with non-endpoint applications, such as Netflix or YouTube. The way the attack works is when you click “Sign-in with Microsoft”, you are given a token that lasts 15 minutes to complete the authentication flow. This makes sense given it’s designed for devices other than your laptop.

Traditional phishing campaigns must generate the token before sending a phishing email, which can limit the infection window. According to Microsoft, EvilTokens' unique approach is to use AI-generated frontends and workflows to create on-the-fly tokens via a hyper-optimized phishing page, thereby extending the window to the full 15 minutes, since tokens are generated only when the victim interacts with the attacker's infrastructure.

Tracking an OtterCookie Infostealer Campaign Across npm by Alessandra Rizzo

In the latest evolution of Contagious Interview/WageMole and FAMOUSCHOLLIMA-aligned threat actors, Panther security researcher Alesandra Rizzo tracks an open-source supply chain attack that results in an OtterCookie infection, followed by the exfiltration of developer secrets and machine configuration files. To me, there are two interesting findings that showcase the evolution of DPRK-nexus threat actors.

First, they are heavily abusing Vercel services, making it easy to stand up and rotate attacker infrastructure used as exfiltration points. Secondly, the OPSEC trickery around dotted Gmail email addresses, such as t.e.ch.y@detectionengineering.net, allows them to get a little more use out of the emails, since Gmail ignores dots when receiving email on behalf of users, whereas other services like npm do not.

ClickFix technique uses Script Editor instead of Terminal on macOS by Jamf Threat Labs

When I first learned about the ClickFix infection technique, I couldn’t believe that people would copy and paste terminal commands from a website into their Terminal. I scoffed at people falling victim to it, exclaiming that it would never happen to me because I’m a security person. I then proceeded to install a package manager by copy pasting a bash command into my terminal:

The technique works because the industry has collectively settled on the ease of installing software via a copy-paste command. So, as vendors like Jamf and Apple began to catch up by deploying mechanisms to detect this behavior, threat actors adjust and continue the cat-and-mouse game.

In this post, Jamf Threat Labs uncovers a ClickFix campaign they discovered that combined the social-engineering aspect of a lookalike website with an older technique: AppleScript URIs. According to the researchers, victims are presented with the phishing website, and instead of copy-pasting the command, you click an Execute Button, which runs a native applescript:// feature that launches a Script Editor and prompts the user to run it.

The payload leads to an infostealer infection so not much changes there, but adjusting the delivery and exploiting the trust of victims running these commands are just one UI/UX workflow away from a new infection.

🔗 Open Source

salesforce/url-content-auditor

url-content-auditor scans web content for sensitive data, such as secrets and PII, as well as anything that puts the website's or its users' privacy at risk. It’s smart enough to download video, audio, and documents, extract data, apply some heuristics, and also use LLMs to classify and alert on anything sensitive.

momenbasel/malware-check

Modern static and dynamic analysis toolset for malware analysis. It has an impressive number of analysis engines, including Windows, macOS, Linux, Android, and iOS. It uses a Docker sandbox for its dynamic analysis, so it’s pretty lightweight. It generates findings in the console, JSON, HTML, and SARIF for CI/CD pipeline reports.

416rehman/DeepZero

DeepZero is a research toolset using quite a grab bag of techniques to find vulnerabilities in Windows kernel drivers. Two features stood out to me. One, it uses Semgrep rules on decompiled binaries to find “known vulnerability shapes”, which essentially means it can direct analysis towards interesting findings versus sweeping the whole binary. On the back of the Mythos announcement, it uses DeepAgents from langchain and Vertex AI to triage the Semgrep findings.

momenbasel/htb-writeups

Massive compendium of HackTheBox writeups used for self-learning and exploration. This is super helpful for those who want to explore topics as they work through HackTheBox challenges, or they want to see and read about techniques used during these challenges. It has four interactive tools you can use to query and generate write-ups based on your interests: everything from searching for specific machines, operating systems, and attack paths to a Skill tree that maps out your learning journey.

pandaadir05/snoop

Slick-looking syscall tracer leveraging eBPF versus strace’s ptrace. This is especially helpful if you are using it to research malware or hunt for vulnerabilities in binaries on CTFs. The TUI is quite beautiful and interactive, whereas strace makes me want to cry every time I stare at it.

Welcome to Issue #151 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• I tried to visit my hometown over the weekend, but my flight was canceled before I could leave. I did my first solo road trip in probably years. Maybe it’s an American culture thing, but I didn’t mind the 6.5-hour drive. Lots of music, podcasts, and sitting with your thoughts

• It’s always strange going back to your hometown and seeing how much has or hasn’t changed. For example, it’s almost mid-April, and I drove into snow :(. But pizza & chicken wings are so much better in NY than in New England so I hope that never changes

• I’ve been reading about Daniel Miessler’s PAI project, and I’m quite impressed with the idea of using AI for Personal Augmentation. Rather than having several Claude Code sessions or optimizing ways to integrate into Gmail or Calendars, you can use this almost like an extension of yourself. It learns your motivations, wishes, and tool-stack preferences, and even tries to configure its personality so you enjoy working with it. This is definitely my project for the next several weeks

Sponsor: Nebulock

Automate the Tedious Parts of Your Hunting Workflow

The hardest part of threat hunting isn’t running queries. It’s knowing what to look for, why it matters, and whether your environment is exposed.

Distilling reports, mapping TTPs, and translating into behavioral indicators is where time disappears. Vespyr, Nebulock’s autonomous hunting agent, handles the reasoning layer. Findings are tied to your stack, your data, and your exposure profile, so every result is relevant to your environment and ready for the judgment calls only you can make.

See How Vespyr Works

💎 Detection Engineering Gem 💎

The Implementation Blind Spot | Why Organizations Are Confusing Temporary Friction with Permanent Safety by Chris St. Myers

This is an excellent commentary on the risks in the adoption curve of AI and Agents in security. It’s easy to get overwhelmed by the noise of marketing, fear, uncertainty, and doubt about security. On the one hand, we are hearing about so many companies adopting AI to increase productivity, sell products, and, more often than not, citing its use to justify layoffs. On the other hand, AI doomers claim that this technology will ruin our careers by automating us away. Like most things in life, the answer is probably somewhere in the middle, but we need to make sure we understand the risks.

We are all fortunate to be standing on the shoulders of giants. We know what a good security product, alert, or workflow looks and feels like. AI is too nascent for us to forget how much we’ve had to practice learning our craft with deterministic tools like Wireshark, the command line, and SIEMs. St. Myers warns, though, that we are at risk of forgetting. He compares and contrasts this with the massive adoption of technologies like the cloud, where we retained the analytical capabilities of security people and anyone in technology, because it was a deterministic shift in architecture. We still needed to understand and synthesize information to help automate tasks.

We are not just changing the pipes; we are changing who (or what) processes the data.

But, for AI, it’s non-deterministic, and that’s by design. And the ‘who’ in the quote here is important. St. Myers calls this risk the “cognitive rust belt”. We aren’t farming out architecture, building, or repetitive tasks to AI; we are farming out analytical capabilities. It’s a gradual hollowing out of analytical capabilities, as if we were all handed a junior analyst to synthesize data for us, and all we read are prompt responses.

Here’s how it relates to detection and response:

• We’re building out increasingly complex detection technology, but we risk losing the understanding of why those detections matter, and how we can investigate when they fail

• For AI-generated triage, are we slowly removing the “approved by an analyst” workflow? What parts of D&R will we lose agency to AI?

• If we solve SOC analyst burnout with AI, which is great, what do we lose in the process? How else can they learn the field if they don't sit down and work through alerts?

They have been living inside summaries, not raw telemetry.

These are paradoxes in detection engineering, but honestly, it applies to every place trying to replace or accelerate human analysis with AI. We have to find ways to train and retain this expertise in an analytically rigorous profession. The prompts will be tuned and perfected, direct feedback on results will become more opaque, and we run the risk of understanding the how underneath the hood. When we enter the rust belt, it’ll be harder to trust the output of LLMs without trusting that we have the expertise to judge them.

🔬 State of the Art

I fell in love with Darknet Diaries years ago, probably starting with the Carbanak Episode. It’s cool to learn about the intermix of pure cybersecurity, professional stories, and security-adjacent stories through Jack’s storytelling. In this episode, Jack interviews the CEO of BeatDapp, who first started out as a fraudster in the BlackHat/GrayHat SEO realm. They began as a marketing firm but are now a fraud-prevention platform for the music industry. There are SO many parallels to security.

• Fraud impact is directly measurable to impact (loss prevention), and bad guys are extremely persistent in finding ways to perform fraud

• Many techniques to perform that fraud involve security means, such as compromising individual accounts all the way to compromising streaming services to skim money from payouts

• Detection rules range from basic heuristics to machine learning, and clustering activity is a huge part of finding fraud

I also learned a few things about the streaming platform’s business model after this. Advertisers pay apps like Spotify or Apple Music for ads, and the money goes into a single pool each month. The streaming services then take all the listen counts by artist, sum them, and divide them across artists to create pizza slices (percentages) showing how each contributed to that sum. Then they carve out a portion of the ad revenue to pay artists and divvy up the payments according to those percentages.

So, if you compromise an artist or the streaming services, and you can take money off the top of those payouts, you can make a lot of money.

Fascinating stuff!

A Detection Researcher Mindset by Scott Plastine

It always fascinates me to find posts like this one by Plastine that outline their mental model in how they approach research and detection ideation. Detection ideation typically begins with a news story or a research blog post that (hopefully) contains enough technical detail to initiate the process. Then, you should deconstruct this information into components around capabilities, environmental context, existing coverage, and feasibility. This is easier said than done, so Plastine splits this into seven steps, with, funny enough, the last step being to write the detection.

They first start with understanding the technique and what normal behavior looks like in the context of the attack. A lot of people jump straight into writing rules without properly investigating whether this is even relevant to their environments. If it is relevant and you do understand the attack, you must then see whether you have the necessary telemetry for your rules to fire.

My favorite step in this blog, though, is under “is prevention possible?” A metric we can all obsess over is rule count and coverage, and making sure they go up. More rules is more coverage and more attacks, right? As an industry, I think we need a separate metric that accounts for cases where we remove rules because we implemented a technical control to limit the attack path altogether. Seeing Plastine call this out as a possibility in rule development means teams obsess less about hitting coverage metrics and more about recommending and implementing security controls that make all of our lives easier.

SITF: The SDLC Infrastructure Threat Framework by Wiz Research

We can’t always wait for MITRE ATT&CK to release new frameworks so quickly; many great research and security teams can help fill that gap with their own ATT&CK-style frameworks for everyone in the industry. The SDLC Infrastructure Threat Framework, or SITF, helps solve that gap. Here are some gaps and features they address:

• They list five components of potential victim infrastructure: Endpoint, VC, CI/CD, Registry & Production. You can see these being attacked in every supply chain attack in the last two weeks surrounding Trivy & Axios

• Three stages, Initial Access, Discovery & Lateral Movement and Post-Compromise, connect to ATT&CK, sans post-compromise

• The techniques are specific and actionable. For example, Git Tag Manipulation was used in the Trivy attack as tags were removed and re-added with an orphaned commit on a fork in the attacker’s repo

Each technique has protective controls associated with them, so this is great reference material for those who are trying to harden their supply chain pipelines.

PR3TACK by Atlassian CSIRT

The Preemptive Tactics & Countermeasures Knowledgebase (PR3TACK) is an ATT&CK-style lexicon of tactics and techniques that highlight theoretical or “hard to observe” attacks. It’s a bit hard to understand at first, but once you dig into their matrix, there are some interesting entries. For example, the following collection technique:

There is malware that abuses clipboard content theft, so it makes sense that operating systems have mechanisms to cache history in some fashion. Each technique has a preemptive defense section, and in this case, it states there is no effective way to detect this type of attack due to a lack of telemetry.

It also introduces eight unique tactics that “extend beyond traditional technical compromise into governance, cognition, and sociotechnical domains.” There are supposedly longer descriptions for each one, but it either seems like the website doesn’t have a page to navigate to or my Brave browser is broken :3.

☣️ Threat Landscape

Axios Post Mortem by Jason Saayman

The owner and victim of the Axios supply chain attack last week published a great post-mortem on GitHub issues. Not much new information was shared, but you can tell they took the attack seriously and were an unfortunate victim to a convincing social engineering attack likely led by DPRK operators. They could have taken some steps to prevent this from happening, such as:

• Removing long-lived tokens for publishing out-of-band versions

• OIDC-style publishing to issue short-lived tokens and force releases through GitHub

• Immutable-builds: this can mean many different things, but pinning to a specific version of axios that uses bundleDependencies, for example, can prevent consumers of axios from pulling in updated malicious versions

Even if Axios hardened their build pipeline with the above bullets, th

Attackers Are Hunting High-Impact Node.js Maintainers in a Coordinated Social Engineering Campaign by Sarah Gooding

Following the Axios breach and the subsequent post-mortem above, Socket.dev researcher Gooding collected several notable open-source maintainer posts about how they were contacted by the same threat actors in the same campaign. It’s good to see the openness of many of these maintainers to share their stories. It brings transparency to the situation and a sense of community that they are all in this together. It’s bad to see how wide DPRK cast their nets and have succeeded with at least one victim.

These developers are all self-selecting, meaning many more likely got these phishing emails and Slack invites. I’m unsure if there were any more victims, but I wouldn’t be surprised.

I have to apologize to you all. I listened to lots of podcasts on a long drive over the weekend, and this one stuck with me in particular because of its coverage of the war in Iran. The U.S. military industrial complex has warned of a “Cyber 9/11” event since I’ve been in the industry. The idea is a thought exercise in which a single cybersecurity breach or attack can trigger massive kinetic effects without a nation-state ever leaving its computer screens.

It’s a term that’s been made fun of relentlessly. Nation-states have effectively used these capabilities as spying tools, which they are very good at doing. But, starting with the Russia-Ukraine war, we’ve seen attacks mounted that have crossed that threshold. In Iran, there have been reports of Iranian actors using compromised devices to perform Battle Damage Assessments, as well as using them for targeting for a strike.

This is where I see security being relevant in a more modern environment. The grugq and Tom Uren have an excellent conversation in this podcast on everything from cyber 9/11 doomers to the effective use of cybersecurity as an intelligence weapon in lieu of boots-on-the-ground collection activities.

Germany Doxes “UNKN,” Head of RU Ransomware Gangs REvil, GandCrab by Brian Krebs

I haven’t heard the words UNKN, REvil or GandCrab in many years! The wheels of justice grind slowly but grind fine, and it looks like German authorities are joining the fray, along with UNKN and co-conspirators. For those unfamiliar with REvil, it was the O.G. ransomware gang that moved the cybercrime industry from small-scale attacks for a few hundred to a few thousand dollars to a cartel-like operation that claimed to extort over two billion dollars.

🔗 Open Source

Blevene/structured-analysis-skill

Claude plugin for performing structured analysis techniques used by organizations like the CIA and the U.S. intelligence community. This is super useful for people using Claude Code as a threat intelligence research aid. You can instruct your session to use the plugin or skills for everything from attribution and intelligence writing to malware analysis.

Maybe I’m an intel nerd, but I do think a lot of people or companies who write blog posts on threat research could use a toolset like this as a gut check before they start throwing out wild claims to grab attention.

wiz-sec-public/SITF

Wiz’s repository for their SITF supply chain site is listed above in State of the Art.

elastic/supply-chain-monitor

With all the OSS supply chain attacks happening, I think it’s important for security engineers to become more knowledgeable about the OSS ecosystem. For example, how are new packages published or updated, and where can you get better visibility in the upstream publishing process and into how your organization consumes these packages?

The Elastic Security team made that a little easier with a fully packaged open-source tool that monitors PyPI and npm for new packages and package diffs. It normalizes them and feeds them into a Claude prompt for analysis and subsequent alerting.

ironsh/iron-proxy

To continue the supply chain security awareness story, iron-proxy helps prevent data exfiltration or command and control call-outs by injecting a workload on top of your CI/CD pipeline to do network monitoring and egress blocking. It specifies that it can be used for any workload, so theoretically you can run this on top of a developer container or a cloud machine, but IMHO it should shine in test runners within CI/CD pipelines.

HaxL0p4/L0p4Map

L0p4Map is a network scanning tool with a quite stunning front end. I think something like this would be useful in your network, where it can scan for devices, fingerprint them, and perform basic vulnerability scanning to help you understand how an attacker might probe your network for lateral movement.

Welcome to Issue #150 of Detection Engineering Weekly!

✍️ Musings from the life of Zack:

• We completed a 5-hour back-and-forth car ride over the weekend with mostly chill kids, no car sickness, and even some napping. It doesn’t seem like much but it made the trip wayyy less stressful.

• I skipped last week to take some time off after RSA. Thank you to everyone who came up and said hello to me, as well as to those who hung out at the Detection & Response Happy Hour!

• I finally set up Claude to be an “executive assistant”. It’s been helpful to have it sift through email, Slack, and Calendars and give me the most important things up front. It did take some prompt tuning to separate what matters from what is noise, but I recommend setting this up for people who have a busy personal or work life.

Sponsor: Permiso Security

Every identity. Every environment. Know static posture and runtime behavior.

Attackers aren’t breaking in anymore. They’re logging in. And they’re not just targeting human accounts. Service accounts, OAuth tokens, and AI agents are just as exposed and far less monitored. Most detection tools weren’t built for that reality. Permiso was, which is why we won the 2026 SC Award for Best Threat Detection Technology. See how it works in our product tour.

Take the Tour

💎 Detection Engineering Gem 💎

Building a Firewall ...via Endpoint Security!? by Patrick Wardle

<rant>

Apple and lack of accessibility to secure telemetry: name a better duo. Jokes aside, I have such a love-hate relationship with Apple products. I use them for everything. The ecosystem is SO good. I can’t imagine not working on a MacBook, surfing the Internet, taking calls on an iPhone, or sporting around my AirPods Max. They integrate their technologies so well and make them easy to use across devices, and if my parents can figure out how to use them (sorry, Dad & my in-laws), then you know they do something right.

What infuriates me is how opaque they choose to make their devices to researchers, security tinkerers, and security teams at companies who pay Apple a lot of money. There are myriad nation-state threat actor cases in which a sophisticated exploit runs against an Apple device, and victims are unaware of what happened. In every case above, detection required either an external forensics lab (Citizen Lab, Amnesty Tech), corporate network monitoring catching anomalous traffic (Kaspersky), or a specialized third-party iOS monitoring tool (iVerify). Apple’s own platform produced no victim-facing signal.

</rant>

Luckily, we have the macOS researcher community, and Patrick Wardle is one of the lead researchers in this cause. Wardle routinely reverse-engineers macOS releases to discover new capabilities and features to share with the wider community. In this latest macOS 26.4 release, Apple’s built-in Endpoint Security product got some updates. This didn’t come with much (any) documentation, so Wardle built a harness to explore new data types within the framework and determine how to implement them. There were six new event types added with unhelpful names, such as ES_EVENT_TYPE_RESERVED_1

So, Wardle built a harness to subscribe to the Endpoint Security event stream and filter out the event types listed above. He sucessfully subscribed to 3-6, and had such a clever way to print the raw bytes from these subscribed events into essentially what is a hex dump. Instead of parsing each byte, he fed his methodology (plus the dumps) into Claude Code, and it found out they were network logs:

One thing I learned from Wardle is that these event logs send two events to a subscription: an AUTH and a NOTIFY event. The AUTH allows you to allow or deny the event. So, in this case, imagine cURLing a suspicious domain from a ClickFix attack and your Endpoint Security event software denies it. The NOTIFY event is for logging and might be useful for on-device correlated alerts in an EDR or for sending it to your SIEM.

I am really starting to enjoy the macOS detection & response capabilities coming out, and I am hopeful that the research from the small, tight-knit macOS security community can bring them to the masses.

🔬 State of the Art

The C2 Trap by James Rowell

Shiny object syndrome is a phenomenon I see a lot in security, and it can dangerously bias your work towards what is new and trendy. A classic example of this is a security operations team reading threat intel reports on nation-state activity and trying to write rules to catch the latest TTPs. What they may not realize, for example, is that this nation-state targets technologies that the team rarely uses, or they don’t fit into the threat actor’s target set due to their size or industry. We love our shiny objects!

In this post, Rowell describes how detection engineers can fall victim to this syndrome by targeting the wrong parts of the MITRE ATT&CK chain. One of my favorite quotes they use here:

The first mistake that teams make is to treat C2 and exfiltration as if they are specific behaviours. They are not. They are outcomes.

This is super concise for a number of reasons that Rowell points out. First, the “outcomes” portion of this quote means you can’t stop exfiltration without understanding the underlying behaviors. It begs the question: when an actor successfully exfiltrates data from your environment, how did they do it? Was it uploading to a cloud service, a good old-fashioned SSH session, or perhaps emailing a large zip file to their inboxes?

Rowell challenges readers to move to the left of exfiltration if you have to start somewhere. A lot of things have to go right for the attacker in order to get to this stage. So, focusing on behavior chokepoints in persistence, privilege escalation, and lateral movement can meaningfully reduce the complexity of your rule backlog. It’s also likely that you have a limited set of attack paths for data exfiltration, whereas there are many ways to exfiltrate data, as I stated above.

Detection via Deception — Using your SIEM as a Free Deception Platform by Regan Carey

Honeytokens are widely regarded as a low-cost, effective detection tool for identifying threat actors targeting your environment. Thinkst Canary is the prime example of a company that creates this technology, and you can use their canaries for free across a variety of technologies. In fact, it’s a great learning experience for those building detection labs or securing their home networks to deploy these and watch threat actors try to use them, especially if you intentionally expose them.

In this post, Carey does a great job of framing some of the issues people have with deploying canarytokens and calls out some misconceptions behind deception-based alerting and honeytokens. First, the technological barrier to using honeytokens isn’t low, so you just have to find a way to deploy them and send any corresponding alerts. Second, people may believe that their environment isn’t complex enough to warrant using honeytokens, when in fact they may be better for small organizations and environments due to the cost of entry for security products.

They round out the post with an example of using MITRE Engage to deploy and monitor honeytokens leveraging native Azure, Sentinel, and KQL functionality.

macOS EDR Telemetry: A Structured Framework for Evaluating Endpoint Visibility by Kostas Tsialemis

I first covered Kostas’ EDR Telemetry Project in October 2024, and it’s grown so much since then! The project provides a framework for benchmarking the detection & response capabilities of various EDR vendors across Linux and Windows. With this release, the team added a macOS benchmark. The framework comprises 16 categories and 58 subcategories across 8 EDR projects.

I’m glad to see them tackle macOS: infostealers have been a popular target for cybercriminals. Apple built Endpoint Security (ES) for security vendors to subscribe to, similar to Windows ETW or eBPF on Linux. The issue with ES is it’s super noisy, and because it’s not technically an inline hook, there’s not the same level of inline blocking as you’ll see on Windows and Linux. They released a companion query-generation tool that people can run while comparing their EDRs or security tooling against the framework.

SecCompare by Mark Manning

Long time friend of the newsletter, Mark Manning, is one of the leading researchers in container and Kubernetes security. He recently gave a talk at BSidesSF that surveyed various Linux security guardrail tools and their corresponding threat models. Within the talk, he showed his tool, SecCompare, which helps people understand SeccompBPF filters, and compare and contrast expected behavior from filters you generate versus baselines. These can get tricky as there are peculiar attack paths among many different Linux syscalls that can bypass a seccomp filter without you realizing it.

He’s got a sick Linux syscall table lookup with information on each syscall as well as labels around how you can abuse them. The interactive “how do containers work” demo is sick, too.

Slightly safer vibecoding by adopting old hacker habits by Halvar Flake

Vulnerability Research GOAT Halvar Flake released a timely post on his setup for coding in the wake of all of the TeamPCP supply chain attacks over the last few weeks. It’s a short but sweet post that starts to bring up “old” ways of doing things I learned when I first started in security. A basic setup involving a rented server, SSH, and a clear separation between your physical computer and your coding machine seems secure because you don't have any personal information, keys, or crypto wallets on the rented server.

I’ve been thinking a lot about old concepts from 14-15 years ago. We have been in container nirvana for the last 8 years or so, and the security boundaries are hard to nail as you begin to develop more and more in them. You load keys inside containers, push them to remote registries, and build them alongside your code. Each step of the deployment pipeline becomes an attack vector, and people aren’t paying attention to the endpoints writing this code, which are also part of the pipeline.

I can’t wait for Vagrant to come back, as it was my favorite tool for years and years, and virtual machines are way more “secure” by default. Now it’ll just be containers inside a local VM 😂.

Sponsor: Blu Raven Academy

Threat Hunting + Detection Engineering, Powered by Advanced Analytics

Master practical threat hunting and detection engineering through hands-on training with advanced analytics, real-world scenarios, and exercises designed for defenders who want skills they can apply immediately.

Start Learning

☣️ Threat Landscape

Emerging Threat: The Open Source Supply Chain Ecosystem is Front and Center

I take one week off from writing this newsletter, and now the software supply chain is on fire! I am linking posts to several compromises over the last 2 weeks that fall into two buckets: Axios & Nation-State Activity and TeamPCP & Cybercriminal Activity. This is becoming more significant from a detection perspective because the expertise required to understand how threat actors carry out these attacks is becoming more prevalent in our detection & response community.

The other component to call out is that the impact of these attacks extends beyond cryptominers; they can serve as primary, secondary, and tertiary initial access vectors as the bad guys work through the exfiltrated code, secrets, and infections sourced from these compromises.

I am linking two stories from $DAYJOB, so full disclosure, they are my colleagues and excellent researchers.

Compromised axios npm package delivers cross-platform RAT by Christophe Tafani-Dereeper

The first set of news, as of me writing this (Mar 31), is that the very popular Axios library for JavaScript was compromised. Axios has over 3 million weekly downloads, and these downloads range from individual developer laptops, CI/CD systems, and production environments. The threat actor compromised the owner’s account and inserted a backdoor in the dependency list. This malware had payloads for macOS, Windows & Linux. Two versions of the compromised package were released using the NPM publishing token and were exposed for around 4 hours.

Christophe’s analysis and timeline are excellent here. He covers each payload and the second-stage RAT, and also notes that this attack didn’t necessarily work in many environments due to errors in the initial loader logic.

Inside the Axios supply chain compromise - one RAT to rule them all by Ruben Groenewoud, Samir Bousseaden, Salim Bitam, Joe Desimone, Colson Wilhoit & Andrew Pease

This post from the Elastic Security Research team helps shed light on the malware payloads in the second stage of the Axios attack. They said that the RAT dropped on this stage shares a “significant overlap” with WAVESHAPER, a malware family tracked by Mandiant that is connected to a DPRK-linked threat cluster. I thought this was a helpful table to show the details of this campaign compared to WAVESHAPER data:

LiteLLM and Telnyx compromised on PyPI: Tracing the TeamPCP supply chain campaign by Nick Frichette, Sebastian Obregoso, Christophe Tafani-Dereeper & Emile Spir

The fallout from the Trivy compromise led to several package compromises, and LiteLLM & Telnyx were among the bigger ones.

The timeline above helps explain my take at the start of this section, a round primary, secondary, and tertiary compromises. Specifically, look at the BACKDOORED sections of each box. Given the many integration points in a CI/CD pipeline, rotating one credential doesn’t imply that another access point could provide a backdoor.

TeamPCP Supply Chain Campaign by Rami McCarthy

There has been some amazing research on this campaign by several researchers and vendors, and Rami led the way throughout. If you need a quick reference blog that’s easy to navigate, he built a beautiful website outlining the campaign so you can do just that. There are references to the timeline itself, IOCs, the payloads, unanswered questions, myths, and a nice playlist with songs for each part of the compromise!

🔗 Open Source

agentshield-ai/sigma-ai

Sigma ruleset for detecting malicious activity within agent behavior. It’s more of a pure detection-and-alerting toolset than an EDR, compared to some of the “Agent EDRs” I’ve linked in the newsletter. It’s listing 42 different Sigma rules, which is an impressive set for agent threat activity.

cisco-ai-defense/defenseclaw

OpenClaw plugin that provides an AI gateway and several governance functions to help secure your OpenClaw deployment. What’s cool is that it hooks OpenClaw to scan every skill, MCP servers, and plugins before they are installed and used by the agent. It also has some code security scanning capabilities, as well as run-time monitoring through the gateway.

awslabs/threat-modeling-mcp-server

Locally-run MCP server that provides threat modeling capabilities and tool calls. It uses the STRIDE framework to contextualize the application or code you are modeling and attempts to learn the application's business context before providing recommendations.

alicankiraz1/Codex-Sentinel

Yet another awesome-* repo for skills on Codex. It’s similar to the threat-modeling MCP server above, but primarily relies on skills to help shape secure-by-design coding practices and inject security tests into code artifacts.