Threshold signature schemes, a form of multi-party computation (MPC) that lets a set of parties sign together without any one of them holding the key, are increasingly deployed inside trusted execution environments (TEEs). The combination is intended to amplify security for sensitive computations: MPC distributes trust across multiple independent parties, while TEEs root trust in the hardware manufacturer and its attestation infrastructure. But subtle issues can arise when running an MPC protocol inside a TEE without accounting for the untrusted host: for example, a malicious host could roll back the filesystem state after a threshold signer deletes a used pre-signature, causing the signer to reuse their nonce share and disclose their private key share.

So is this combination worth it? Provided you treat the TEE as a defense-in-depth layer rather than a substitute for a sound protocol, the answer is yes. This blog post discusses what TEE attestation can and can’t fix in MPC deployments, explores the pitfalls we see most often in audits, and covers best practices, such as incorporating strong attestation processes and binding them to the MPC parties’ identities.

MPC: Security that depends on participant behavior

Before diving into how TEEs and MPC interact, we need to understand what MPC means and what security guarantees it offers. MPC is a cryptographic technique that allows multiple parties to jointly compute a function over their private inputs without revealing those inputs to each other.

The security of MPC protocols depends critically on assumptions about participant behavior. The cryptographic literature uses two primary security models:

Semi-honest (honest-but-curious) security: In this model, all participants follow the protocol exactly as specified, but they may try to learn additional information from the messages they receive during the protocol execution. Participants can try to learn more than they should, but they don’t deviate from the protocol specification.

Malicious security: This stronger model assumes participants may deviate arbitrarily from the protocol. A malicious participant might send incorrectly computed values, use wrong inputs, abort the protocol at strategic moments, and behave in ways designed to compromise security or learn private information.

This distinction is important in the context of TEEs. If a TEE attestation can cryptographically guarantee that all parties are running the correct protocol implementation, it effectively elevates semi-honest protocols to provide malicious security guarantees (at least against certain classes of attacks, as we’ll discuss later). But before delving into the details, let’s discuss how TEEs work.

TEEs: Three core security guarantees

TEEs are secure areas within a processor that provide hardware-based protection for code and data, even from privileged software like operating systems or hypervisors. TEEs offer three core security guarantees:

Confidentiality: Data and code are encrypted in memory and accessible only from within the TEE. This ensures that even privileged system software cannot inspect the contents of the secure computation.

Integrity: The data and code are protected from tampering. Any attempt to modify the TEE’s memory or execution state from outside should be detected.

Attestation: Remote parties can cryptographically verify what code is running in the TEE. This allows external verifiers to gain assurance about the computation being performed and the legitimacy of the TEE without trusting the host system.

This last property is particularly crucial for building distributed systems with TEEs.

How TEE attestation works

The attestation mechanism is at the heart of TEE security. When a TEE is manufactured, it’s provisioned with a private key and a corresponding certificate that chains back to a root certificate held by a trust anchor (typically the manufacturer).

When an attestation is requested, the TEE takes measurements (cryptographic hashes of the TEE software, configuration, and hardware state). These measurements are bundled into a “quote” (a manifest of these cryptographic hashes), which the TEE then signs with its private key.

To verify these attestations, a number of checks need to be performed. However, the checks necessary in the verification process aren’t uniformly defined and are somewhat vendor-specific. For instance, Intel’s TDX documentation states:

TD Quote Verification is the process by which a TD Quote is verified in a remote attestation flow. This verification can be done by any party and the checks performed are defined by this party.

This places significant responsibility on developers, who might not fully understand what checks are required to ensure the security of the TEE deployment.

At a minimum, a proper verification process requires:

1. Verifying the quote signature
2. Verifying the certificate chain back to the trusted root
3. Verifying the actual measurements against known-good values (often stored in binary transparency logs)

This means deploying TEE-based systems involves not just cryptographic verification but also reproducible builds and binary transparency infrastructure. The measurements in the attestation quote are simply hashes; they don’t inherently indicate whether the code is correct or malicious. To verify that the TEE is running the intended software, what is required is a trusted source of reference measurements. Reproducible builds ensure that anyone can compile the same source code and arrive at identical binaries (and thus identical measurements). Binary transparency logs provide a tamper-evident record of what measurements correspond to what software versions, allowing verifiers to check that the TEE is running legitimate, audited code rather than a compromised variant. This piece is often overlooked in TEE deployments.

The trust model clash

Here’s where things get interesting. Multi-party computation is fundamentally about distributing trust among multiple participants. No single party should be able to compromise the computation. TEEs, in contrast, centralize trust in the hardware manufacturer that controls the root certificate.

Another important threat model shift to consider is the fundamental inversion of traditional security assumptions in TEEs. Historically, we trusted the host operating system and built our security models on that foundation. TEEs flip this on its head: they’re virtualized environments where the guest (the code running inside the TEE) is trusted, but the host system is explicitly considered untrusted and potentially malicious.

This shift creates new attack surfaces that didn’t exist in traditional computing models. While the TEE protects the guest environment from the host, the host still controls critical operations like I/O, memory management, and CPU scheduling. The host can deny service, manipulate timing, or attempt rollback attacks, leading to subtle breaks of MPC protocols.

What TEEs can (and can’t) fix in MPC deployments

Combining these technologies can be beneficial, but one must be careful about the security claims that can be made. The security gains are real only if the TEE implementation is sound and the attestation process verifies the right properties. When TEEs attest to incomplete measurements or fail to verify critical components, the attestation provides false assurance. The actual security posture may even be weakened by the added complexity.

In an attempt to narrow down what TEEs can fix in MPC implementations, we’ve identified the common categories of MPC issues we frequently encounter in security audits:

1. Ambiguous encoding in hash functions (e.g., “YOLO” is not a valid hash construction)
2. Misbehaving participants
3. Missing parameters in Fiat-Shamir transforms (e.g., Coordinated disclosure of vulnerabilities affecting Girault, Bulletproofs, and PlonK)
4. Missing input validation (e.g., Breaking the shared key in threshold signature schemes)
5. Side-channel attacks
6. Protocol-level issues (e.g., Friends don’t let friends reuse IVs)

Arguably, if TEEs were perfectly secure, they could effectively mitigate the first four categories. When attestation and measurement processes are correctly implemented, each party gains cryptographic assurance about what code the other parties are running. This is powerful: if a party can verify that all other participants are running the exact protocol implementation expected (with no modifications or protocol deviations), then they know other participants will follow the protocol correctly and won’t misbehave by sending malformed messages. Essentially, correct attestation eliminates entire classes of implementation vulnerabilities by ensuring protocol compliance on both the sender and receiver sides. However, perfect security doesn’t exist in practice. Real-world TEE implementations have vulnerabilities, attestation processes can be incomplete or incorrectly verified, and the adversarial host environment introduces attack surfaces that are not traditional to MPC deployments.

A cautionary tale: Rollback attacks with threshold signatures in TEEs

Consider threshold signature schemes, which often use pre-signature optimizations. In Schnorr or ECDSA threshold signatures, participants can precompute nonce commitments independently of the message, speeding up the online signing phase. These precomputed nonce commitments are called pre-signatures, and the reuse of a pre-signature value by a participant leads to the leak of their private share (akin to how nonce reuse in traditional Schnorr and ECDSA signatures leads to private key recovery).

It might be tempting to run such an MPC protocol inside a TEE enclave to mitigate these types of attacks. But subtle issues can arise. In some implementations, signers store pre-signatures to disk and delete them after use (to limit the possibility of reuse). If that application were deployed to a TEE in a trust model where the host is potentially malicious, the host could roll back the filesystem state after the signer deletes a pre-signature file. When the TEE later fetches the pre-signature contained in that file, the signer essentially reuses their nonce share, which leads to a disclosure of the private key share. This is also discussed in the blog post A trail of flipping bits.

Common TEE pitfalls

Beyond the rollback issue, several other pitfalls frequently appear:

Incomplete attestation measurements: The security of TEE attestation hinges on measuring the right things. An attestation quote contains hashes of code and data, but project developers determine what gets measured and what gets excluded. If critical components aren’t included in the measurements, an attacker can modify those components without detection. This is particularly risky because the attestation appears cryptographically valid, but it’s attesting to an incomplete picture of the execution environment. For example, installing tools in an enclave from remote sources at runtime (e.g., using curl) pulls in code that was never measured; an attacker gaining access to that source can substitute their own and gain code execution inside the enclave.

A real-world example demonstrates this risk: a vulnerability in Meta’s WhatsApp Private Inference revealed that the attestation process didn’t include certain configuration files, enabling an attacker to inject malicious shared libraries with the LD_PRELOAD environment variable, potentially compromising the TEE while maintaining a valid attestation quote.

Missing verifications: The attestation process requires multiple verification steps, each critical to the security guarantee. Verifiers must check the quote signature, validate the certificate chain back to the manufacturer’s root certificate, confirm that the measurements match expected values, etc. Skipping any of these steps breaks the security model. Because verification procedures vary across TEE vendors and aren’t always well-documented, implementations sometimes omit crucial checks.

Lack of hardening: A TEE protects whatever runs inside it, including components that may not be needed. Redundant kernel drivers, overly permissive kernel config flags, stray ACPI table entries, unnecessary systemd services and timers, and weak entropy sources all increase the attack surface inside the trust boundary. Trim the image down to what the project requires, harden what remains, and measure it. For platform-specific guidance, see our notes on the AWS Nitro Enclaves attack surface.

Data availability and backups: Enclaves have limited persistent storage, so backups and recovery data typically live outside the trust boundary. Authenticated encryption provides confidentiality and integrity but not freshness, so a malicious host can serve a stale backup to force a rollback, as discussed earlier in our threshold signature example. More concretely, one recurring failure is a guest trusting attacker-controlled metadata that lives on disk: the disclosure of several vulnerabilities in LUKS2 disk encryption for confidential VMs shows that a malleable, unvalidated LUKS2 header lets an attacker with disk write access swap in a null cipher, leaving the guest reading and writing secrets on an effectively unencrypted volume.

Side-channel and physical attacks: Because the untrusted host controls the TEE’s execution environment, it can perform various side-channel attacks to extract information about the guest’s computation. These can leak sensitive information and break a TEE’s confidentiality guarantees. Similarly, researchers regularly publish physical attacks on TEE hardware. For example, the recent “TEE fail” attack allows an attacker to break the security guarantees of Intel TDX and AMD SEV-SNP by using a memory interposition device that lets them physically inspect all memory traffic inside a DDR5 server. The more recent DDRop attack goes even further: an interposer costing under $200 silently drops DDR5 writes, so the processor keeps reading stale data that still decrypts correctly, breaking the integrity of Intel TDX, Scalable SGX, and AMD SEV-SNP. Memory encryption without freshness fails the same way disk state does in the rollback example above.

Centralization risks: When MPC aims to distribute trust but TEEs centralize it in the manufacturer and its attestation infrastructure, the security goals of the protocol may be undermined. If all parties in an MPC protocol run their TEE nodes on the same TEE vendor’s hardware or if they use the same cloud provider, trust becomes centralized in that provider (for example, by way of the attestation chaining back to that vendor-operated root of trust).

Insecure defaults: Be aware of some of the limitations that vendor-provided verification tools have. For example, Intel tools allow known-vulnerable firmware versions to pass the remote attestation process for one year after public disclosure. Users might want to implement their own security version number validation on top of the defaults to enforce stricter deadlines.

Best practices for combining TEEs and MPC

If you’re building systems that combine these technologies, consider these recommendations:

Implement strong attestation processes: Bind attestations to the identities of MPC parties and verify them comprehensively. This means not just checking that an attestation is valid but also confirming that the specific party you expect to be running the code is actually the one presenting the attestation.

Terminate peer-to-peer communications inside TEEs: This provides end-to-end protection for party communications. By establishing TLS or other encrypted channels directly between TEEs (with endpoints inside the secure enclaves), you prevent the untrusted host from observing or tampering with protocol messages.

Perform comprehensive verification of the attestation and measurements: Refer to your specific TEE vendor’s documentation for the necessary checks to be performed, and ensure every verification step is implemented correctly, tested, and reviewed.

Write constant-time code: This helps prevent timing side-channel attacks. Even within a TEE, timing variations in your cryptographic operations can leak information to a host performing careful measurements. Use constant-time implementations for all security-critical operations processing secrets.

Consult vendor-specific guidance: Each TEE platform has unique characteristics, limitations, and recommended practices. Vendor documentation often includes details about platform-specific threats. Stay current with security advisories and updates, as TEE security is an active research area with new vulnerabilities regularly discovered. Follow best practices documentation, like our guidance for AWS Nitro Enclaves.

Use multiple TEE vendors: For defense in depth, consider running MPC protocols on TEEs from different manufacturers (and/or cloud providers) to avoid centralizing trust in a single vendor. While admittedly challenging in practice since it requires additional operational complexity, this diversity can be essential for high-security applications where trust distribution is critical.

Layered trust beats either layer alone

Combining MPC and TEEs is not automatic security, but when done correctly, it represents a defense-in-depth strategy that’s stronger than either technology alone. TEEs can effectively mitigate several categories of MPC vulnerabilities by cryptographically guaranteeing correct protocol execution. Meanwhile, MPC’s distributed trust model reduces the impact of TEE manufacturer centralization and may provide security even if individual TEE instances are compromised.

However, this benefit only materializes when both layers are correctly implemented. New vulnerabilities can emerge from their interaction, as we saw with the rollback attack example. Successful deployment requires following established best practices, implementing thorough attestation verification, and carefully reviewing the implementation and deployment procedure.

The intersection of these technologies represents the cutting edge of secure computation, but only if we approach it with a clear understanding of the tradeoffs involved. If you’re deploying MPC on TEEs, our cryptography team reviews these exact systems, so get in touch!

Born out of academia and raised in corporate IT departments, the Security Assertion Markup Language (SAML) authentication protocol continues to be a staple in these organizations. However, it’s time for it to retire. With the rise of software-as-a-service (SaaS) companies in the late aughts, IT departments needed a way for users to authenticate to many new web services. SAML and the burgeoning single sign-on (SSO) industry fulfilled this need. However, SAML is being crushed under the weight of its own complexity. It’s time to deprecate it and move on to modern alternatives like OpenID Connect (OIDC). In this post, I will explore the design-by-committee origin of SAML, its progression through the ranks in academic and corporate environments, its slow disintegration at the hands of the security research community, and its (hopeful) deprecation in favor of newer protocols.

What’s insidious about SAML is that it really is mostly straightforward to understand, but it’s built on a foundation of sand, bone dust, and ash; it works … if you assume XML signature validation is reliable. But XML signature validation is deeply cursed, and is so complicated that most fielded SAML implementations are wrapping libxmlsec, a gnarly C codebase nobody reads.
— Thomas Ptacek, 2023

SAML and the birth of the SSO industry

Wikipedia tells me that “SAML is an XML-based markup language for security assertions.” It was created in 2002 by the Organization for the Advancement of Structured Information Standards (OASIS) Security Services Technical Committee (SSTC). Okay, we’re not off to a great start by modern standards. XML, despite having some redeeming qualities, is quite complex compared to newer alternatives like JSON, but we’ll get more into that later. Further, a committee of subcommittees having meetings is a recipe for “kitchen-sink” protocol design (e.g., waterfall methodology, big design up front, etc.). And sure enough, we’ve now jammed four (!) XML-based security protocols into one:

… the following intellectual property was contributed to the SSTC:

• Security Services Markup Language (S2ML) from Netegrity
• AuthXML from Securant
• XML Trust Assertion Service Specification (X-TASS) from VeriSign
• Information Technology Markup Language (ITML) from Jamcracker

— SAML: History

However, the desire for such a protocol was undeniable. As the internet shifted from Web 1.0 in the 90s to Web 2.0 in the early aughts, users and organizations needed an easy way to authenticate to many new web services. Academia was the biggest driver of this movement, although not the only one: Central Authentication Service (CAS) in 2002 at Yale, Shibboleth IdP in 2003 by Internet2, a consortium of research universities (including my alma mater), ADFS in 2003 by Microsoft, and simpleSAMLphp around 2007 by Uninett, a state-owned Norwegian company with close ties to academia. All these authentication projects eventually supported SAML in one way or another. Like ARPANET before it, universities were at the forefront of internet development and were the earliest consumers of web services. Once this base layer of protocol availability and nascent academic proving ground was established, the commercial industry took it and ran toward a multibillion dollar industry.

The SSO, identity, and authentication provider industry was also starting up in the early aughts, but really came to fruition a few years later: Ping Identity (2002), OneLogin (2009), Okta (2009), and Duo Security (2010). These companies were essentially built on the SAML protocol with the exception of Duo, who would introduce their first SSO product in 2015, which is where I come into the story. I worked on Duo’s first on-premises Access Gateway product (DAG), which was built on simpleSAMLphp and, obviously, the SAML protocol. It’s where I became intimately familiar with the SAML protocol and spent many years of my life digesting its lengthy specifications. I was there when Kelby Ludwig found the XML comment bypass, but we will get into various attacks and SAML deficiencies later. Suffice it to say, the SSO and authentication provider industry was booming, and much of it was built on the SAML protocol.

A crack in the armor

XML signature wrapping (XSW) attacks are the proverbial arrow to SAML’s heel. While there was earlier security research into both signature wrapping (2005, 2008, and 2009) and SAML (2008), I consider the godfather of it all to be “On Breaking SAML: Be Whoever You Want to Be” (2012). It tested theory against practice and resulted in an automated way to check for XSW attacks. This was our north star when implementing the DAG. It was the reason we chose simpleSAMLphp as our building block. PHP, especially at the time, was not exactly known for its security track record, but simpleSAMLphp’s spoke for itself. simpleSAMLphp was resilient to XSW at a time when nobody really knew what that was:

Despite being front and center in this 2012 paper, XSW is still present today. If we know the bug class, then why can’t we fix it? But before we get into SAML’s flaws we first have to consider the shaky ground it was built upon: XML.

XML is no slouch when it comes to a (lack of) security track record. These bug classes would have been more familiar to a developer in the 90s, but nonetheless are still present in XML today: XXE, entity expansion (“billion laughs”), DTD retrieval (SSRF), XPath/XQuery/XInclude/XSLT/CDATA injection, and more. A SAML library needs to handle all these bug classes before even getting to the actual SAML functionality.

In addition to security bug classes, there’s also the sheer complexity of XML when compared against something like JSON. In XML you have tags, elements, attributes, comments, namespaces, markup versus content, schemas, CDATA, DOCTYPEs, and more. In JSON, you essentially have keys, values, objects, and lists. Complexity is generally at odds with security, and this is one reason I consider SAML to be a fractal of bad design.

A fractal of bad design

SAML provides ample opportunity to learn about protocol design. In this section, I will cover five flaws that I consider to be fatal to the long-term viability of SAML as an authentication protocol. These flaws can also be used when designing new authentication protocols. That is, you can either sidestep the flaw, or take its inverse and attempt to bake that into the protocol.

Built on XML

As mentioned above, SAML is built on XML, and XML is complex, but it’s not the committee’s fault. XML is what they had at the time, and it’s what people used. JSON was “discovered” in 2001, but this was right around the time the SAML committee was meeting, and they’d be unlikely to design an authentication protocol around an experimental new format. Especially when it caters to JavaScript and you write a lot of Java.

One could design a quantitative complexity measurement for XML versus JSON or SAML versus JWT/OIDC (e.g., spec/RFC word count, spec/RFC normative word count, etc.), but that would require a blog post or paper all to itself. In the interest of staying on topic, I will refrain from doing that here, and suffice to say that XML is significantly more complex than something like JSON.

Canonicalization

Canonicalization (C14N) is what you do when you want to take the wild mess that is XML, compute a hash of it, and get consistent results. In other words, if the SP and IdP cannot agree on a consistent representation of the XML data, then the bytes won’t line up, the signatures won’t match, and your authentication fails. However, this is easier said than done.

Canonicalization bugs enabled Kelby’s XML comment bypass in 2018:

Canonicalization is often a precursor to parser differential and/or “round-trip” bugs, which are what most modern SAML attacks use:

• Coordinated disclosure of XML round-trip vulnerabilities in Go’s standard library (2020)
• Securing XML implementations across the web (2021)
• Abusing libxml2 quirks to bypass SAML authentication on GitHub Enterprise (2025)
• Sign in as anyone: Bypassing SAML SSO authentication with parser differentials (2025)
• SAML roulette: the hacker always wins (2025)
• The Fragile Lock: Novel Bypasses For SAML Authentication (2025)

Enveloped signatures

Enveloped signature concerns are a not-too-distant cousin of canonicalization. In short, if you’re trying to insert the signature into the data payload that you’re signing, you’re going to have a bad time. Let’s compare and contrast JWT and SAML in this way:

In the JWT example above, the blue signature is detached from the JSON payload and delimited in the JWT with a period (“.”). In the SAML example, the Signature element is inserted (“enveloped”) in the Assertion element. The problem here is that it is very difficult to get a byte-for-byte, canonically equivalent representation of the data when you’re also modifying it! Even more so when you have a complex format like XML and complex canonicalization rules.

“Kitchen-sink” design

This design deficiency essentially transposes to you aren’t gonna need it (YAGNI). It’s not an entirely fair characterization because the portions of the SAML specification that are used in the real-world have changed over the past 20 years (sorry, SOAP and artifact binding). However, the fact of the matter is that 99% of modern SAML implementations use a very similar data shape and subset of the specification. Any given SAML authentication you would encounter in the wild today probably avoids 90% of the specification. This adds significant complexity for largely unused features.

If I was adding SAML support to something new, I’d consider beyond all the standard SAML checks also rejecting any message that doesn’t have the same shape as what Okta, Onelogin, Google, or Shib generates.
— Thomas Ptacek, 2021

Ossification

SAML was designed in a different era for a different time and has not received necessary updates. These concerns will generally be of practical implication rather than theoretical. What I mean by ossification can roughly be enumerated as the following:

1. OIDC assumes HTTP, whereas SAML is transport independent. Sure, SAML HTTP bindings exist and are most commonly used, but they are not required. This affords SAML a certain degree of flexibility, but also means that flexibility must be well defined, must be implemented somewhere, and can contain bugs. SAML came about at a time when HTTP + TLS was not yet the dominant backbone of web service communication, and it has never reconciled with this modern landscape. HTTPS allows OIDC to punt encrypted, trusted communication to the transport layer.
2. OIDC generally assumes a connected network topology, whereas SAML does not. The most common OIDC flow (authorization code) assumes the OpenID Provider (OP) and Relying Party (RP) can communicate directly (OIDC OP/RP == SAML IdP/SP). Sure, OIDC implicit flow with form post exists, but it is very uncommon to see today. Further, SAML has artifact binding for direct communication, but it is also very uncommon. The point is that if the OP and RP can communicate directly then that relieves pressure off of the authentication response payload to contain all the information necessary to make an authentication and authorization decision. This reduces payload size and complexity. The OIDC OP and RP can instead exchange information in a backchannel.
3. OIDC grew organically over the years, whereas SAML was largely designed up front. OIDC encompasses dozens of specifications and RFCs that grew organically over many years. These documents were generally created to solve a specific need rather than trying to anticipate all future needs and building that up front. This is akin to agile methodology versus waterfall, as described in the first section. Consider the following non-exhaustive timeline:
   1. OpenID Connect 1.0 specification published (2014)
   2. JOSE stack finalized for JW{S,E,K,A,T} via RFC 7515-7519 (2015)
   3. PKCE published via RFC 7636 (2015)
   4. PKCE for mobile/native apps published via RFC 8252 (2017)
   5. Device authorization grant for IoT devices published via RFC 8628 (2019)
   6. Demonstrating proof of possession (DPoP) for MFA workflows published via RFC 9449 (2023)
   7. PKCE for SPAs published via RFC 10017 (2026)

I find the historical circumstances interesting too. For example, SAML came of age in an era of VPNs and network segmentation, hence point (2) above. If the IdP or SP was behind a corporate firewall, and it couldn’t speak directly to the other end, then the whole rollout came to a halt and that vendor lost the sales deal. SAML needed to seamlessly account for this situation. Google’s BeyondCorp model and zero-trust architecture flipped this notion on its head in 2014. Additionally, SAML failed to anticipate the mobile, SPA, and IoT revolutions, and had no answers when these technologies arrived on the scene in the late aughts. Even though SAML is still widely adopted in corporate environments, these macroscopic events helped start the long, slow decline of the protocol. Agility and loose coupling enable rapid adaptation in ever-changing IT environments.

All roads lead to OIDC

No protocol is perfect, but in terms of a solution all roads lead to OIDC. As far as I can tell, the only deployment scenario where SAML had an advantage was networks where the SP and IdP cannot communicate directly. OIDC’s implicit flow with form post provides all the same ingredients. This is actually one of the cleanest migration plans I’ve seen available in the industry.

So what can I do if I’m a service provider (i.e., SP) and I’d like to integrate into the SSO ecosystem without SAML? Just support OIDC. Abandon SAML. Apparently Fly.io and Tailscale are already doing it:

We’ve managed to hold the line on OIDC so far. So has Tailscale. If Tailscale can hold the line, given who they’re selling to, I think most orgs can. Really, try to avoid doing SAML. Remember, as a vendor, you’re often competing with companies that don’t do real SSO integration at all.
— Thomas Ptacek, 2024

So what can I do if I’m an identity or authentication provider (i.e., IdP) and I’d like to move off of SAML? Well, depending on your customer count, this may be a long road indeed. But you know how you eat an elephant? One bite at a time. This is a well-worn path in the industry: develop a deprecation plan, communicate it to customers, stop onboarding new customers to SAML integrations, provide existing SAML customers with equivalent OIDC configurations, set a sunset date, and get to work.

SAML had a good 25 year run. It birthed the SSO industry, helped secure untold numbers of authentications, improved the UX of authenticating to dozens of web services, and created billions of dollars of economic impact. We should be thankful to the creators of the SAML protocol. It has provided us with a great case study in protocol design and evolution over a very dynamic period in the tech industry.

If you’d like to read more about historical analyses of security topics, then check out “Marshal madness: A brief history of Ruby deserialization exploits.”

Contact us if you’re interested in a protocol design audit or would like a review of your authentication system.

Security firms have published numerous blog posts describing how they pointed their agent harness at a codebase and found dozens of bugs (we’re one of them). However, these posts tend to focus on agentic code review, which is just one aspect of how we use AI in our security reviews. We want to give a different perspective: before code review even starts, agents now allow us to build custom tooling and formal models that improve the quality and depth of our reviews.

We recently reviewed the Miden VM, a new zero-knowledge VM with its own custom assembly language and almost no developer tooling. To prepare, we spent six months having our agents build an LSP server, a decompiler, a static analysis engine, and a Lean model of the VM executor from scratch. These tools found real security issues, like an unvalidated prover-supplied input that would let a malicious prover forge Falcon signatures and steal funds from Miden account holders. Additionally, the Lean work produced 95 machine-checked correctness proofs, covering a large component of the Miden core library.

Auditing the Miden zkVM

In late 2025, the Miden team came to us to have parts of their zero-knowledge VM reviewed before launch. Part of the review was scoped to cover the Miden core library, which contains a small set of cryptographic primitives written in a custom assembly language called Miden assembly (MASM). This made us genuinely excited, as it was right up our alley: a high-assurance project writing complex cryptographic code in a low-level custom assembly language that we had never seen before. At the same time, it also presented some unique challenges.

To start, the Miden VM implements a stack-machine architecture. This means that each instruction operates on values read from the stack, and the result of the instruction is then written back to the top of the stack. While conceptually simple, this makes code written in MASM challenging to review, since instruction inputs and outputs are read from the stack and are always implicit. Additionally, since the Miden VM is a completely new architecture, very little existed in terms of developer tooling like IDE support, Language Server Protocol (LSP) servers, and linters.

We knew that we had six months to prepare for the review, since the implementation was not yet feature complete, so we asked ourselves: “What could we spend our time and tokens on to make sure that the review would root out as many bugs as possible in the codebase?”

Building all the tools!

Since MASM lacked developer tooling, we started out by asking ourselves what kind of tools we would like to have available when the project started. We typically use VS Code to review code, and syntax highlighting and code navigation are essential for readability and being able to follow data flow throughout a codebase. We needed an LSP server and a corresponding VS Code extension for this, and within a few days we had Claude build a working prototype that provided most of the functionality we wanted: features like syntax highlighting, goto definition, finding code references, and displaying procedure docstrings on hover. With these fundamental features in place, we also decided to add more language-specific features like displaying inline instruction documentation, and stack effects for individual instructions.

Having built the LSP server, we started thinking about other ways to provide high-level semantic information to support manual and agent-driven review. We thought it would be interesting to see if we could provide faithful decompilation for MASM procedures inside the VS Code UI, to help the reviewer quickly understand the high-level control flow and data flow of the procedures they were looking at. For MASM, this is a harder problem than it first appears. Stack machine lifting and decompilation is a well-studied problem, but decompiling hand-written MASM is still difficult for a number of reasons.

1. Most procedures in the core library do not have declared signatures, which means that the number of inputs and outputs must be inferred from context.

2. MASM procedures do not conform to a well-defined calling convention, and the net stack effect of such calls is generally impossible to determine statically. This means that all analysis failures propagate up the call chain.

3. While-loops do not need to be stack neutral, which means that the while-loop condition may occupy a different stack slot in each iteration. This also makes it impossible to map instruction inputs to stack slots for subsequent instructions.

4. Different branches in conditional statements may have different stack effects, which similarly makes stack tracking and signature inference challenging.

This meant that we could not expect to be able to decompile all MASM procedures if we also wanted the decompiled output to be correct. We therefore focused on decompiling a well-defined subset of MASM correctly. During the development of the decompiler, we alternated between using Claude for planning and development and Codex for code review. Whenever we had implemented a new feature, we had agents decompile a randomized set of procedures from the core library and compare the result to the original MASM to look for regressions. Any issues found were added as regression tests to be fixed by the model.

The decompiler represented the single largest effort of the tooling development for this project, with over 100 AI-generated commits over multiple months. The main benefit of this work turned out to be the decompiler’s internal analysis frameworks and intermediate representation, which we could reuse for static analysis, rather than the full decompilation pipeline.

With the decompiler in place, we had access to an intermediate representation of each procedure, with instruction inputs and outputs populated as expressions. This allowed us to bring all the standard static analysis machinery (like data flow analysis) to bear on the problem of finding bugs in the MASM code. We used this to build a number of analysis passes over the intermediate representation, answering questions like the following:

1. Are prover-supplied advice values¹ like remainders and modular inverses properly validated?

2. Are type constraints (e.g. an input being a 32-bit integer or a boolean) enforced?

3. Are local variables initialized across all execution paths?

One way to explore these issues is abstract interpretation. The idea behind this technique is simple. Instead of running the program with real numbers, the analysis tracks the types of values that could be on the stack at each step, like “a 32-bit integer” or “unknown.” It walks through the code again and again until no new information is found. Since it always keeps track of every possible value (with a little extra room), it can never miss a real case. So if a check passes in the analysis, it is guaranteed to hold in every actual run of the program.

We used Claude and Codex to build a general abstract interpretation engine, and then implemented a number of concrete analysis passes on top of it, having the agents switch between development and code review as described above. We also decided to have the agents design and build command-line interfaces for both the decompiler and a new MASM linter to make the new tools available to agent-driven code-review workflows as well.

Finding all the bugs!

During the actual review, these analyses identified over 400 unique locations² where type validation could be improved (all of them reachable from the public API of the library) as well as one high-severity finding.

The high-severity issue was due to an underconstrained advice value in the mod_12289 procedure, which reduces a 64-bit value modulo 12289, with a quotient and remainder provided as advice values by the prover. The quotient is checked to ensure that it is a valid 64-bit value (represented as two 32-bit limbs), but the remainder is never validated before it is passed to the 32-bit instruction u32overflowing_sub. By carefully varying the quotient and remainder to ensure they still satisfy the constraints imposed by the subtraction, we found that it was possible to have mod_12289 return a value that is not the correct remainder. A malicious prover could exploit this to forge Falcon signatures and drain any Miden account controlled by a Falcon key pair.

But what if there are no bugs?

All this work meant that we had a comprehensive set of tools to support both manual and agent-driven code review coming into the audit. However, we’d had a long time to think about other ways to support the manual review process, and we had more ideas that we wanted to test. For example, if the procedures in the core library were implemented correctly and did not contain bugs, would it be possible to prove this using a proof assistant like Lean?

It turns out that the Miden VM is very amenable to formal modeling, since the instruction set is small and most instructions are side-effect free. To formally model MASM, we started by implementing a minimal Miden VM executor in Lean, and had Claude build an automatic translator from MASM procedures to Lean. During the review, we had multiple agents working in parallel to prove correctness of as many procedures as possible across the library. Since the Lean kernel could validate that the generated proofs were correct, we only needed to manually audit the theorem statements to ensure that each theorem proved the right correctness property for the corresponding procedure. To ensure that the theorems were easy to review, we introduced Lean types for field elements and the integer types implemented in the core library. This meant that at a high level, most correctness properties could be expressed in the following form:

If the stack is given by [x1, x2, x3, ..., xn, ...] and we execute the procedure P, then P terminates and the stack is given by [P(x1, x2, x3, ..., xn), ...].

Our agent-driven formal modeling efforts resulted in 95 correctness proofs that covered all of the binary arithmetic components of the core library. This work also identified two subtle bugs that the existing unit test suite did not catch. The first was an edge case in the 64-bit right-rotation rotr, which behaved incorrectly on large inputs greater than the Goldilocks prime if the rotational shift was a multiple of 32. The second was an issue in the 256-bit multiplication wrapping_mul, which dropped caller-owned values from the stack before returning.

Why we couldn’t have done this two years ago

The tools we developed leading up to this engagement, and the Lean libraries and proofs generated during the review, are all side projects that we wouldn’t have been able to dedicate time or resources to just one or two years ago. Projects like these are often highly exploratory in nature, and the end results and potential payoff may be difficult to predict. In practice, this means that it is hard to sell clients on them in advance. However, over the last year, agents became good enough to carry non-essential projects like these with only light supervision, which completely changed the economics of which projects are worth pursuing. Today, a failed side project only costs tokens.

With Miden, the benefits of all this preparatory work were clear. The LSP server and static analysis engine improved our manual review coverage, strengthened agent-driven reviews, and identified real security issues in the codebase that could have led to millions of dollars in lost funds. The AI-generated Lean correctness proofs also improved assurance across a large and fundamental component of the library, allowing the team to continue building on it with confidence. The team has adopted the static analysis engine developed for the review, which means that our side project will now also help secure future updates to the Miden core library.

On a personal note, this has been one of the more interesting projects I have worked on during my years at Trail of Bits. If you find this type of work interesting, or if you are building something similar, get in touch. We want to hear about it!

1Password’s FLAWED report, published on August 6, 2026, gives defenders a misleading picture of AI patching. Its headline says models produced clean fixes only 26% of the time. That figure includes experiments that deliberately instructed agents to apply the wrong fix, along with experiments in which agents could not compile or test their patches.

The report risks making defenders less effective by discouraging them from using technology that could help them fix more vulnerabilities. Teams that take its headline at face value may leave repairable vulnerabilities unaddressed.

We want our work to help defenders fix more vulnerabilities. This post shares real-world data on human and agent patch quality from our consulting projects and Patch the Planet. We’re also releasing two agent skills: post-patch-validation to help agents test fixes, and review-walkthrough to help engineers review them.

How the experiment produces a misleading headline

Our review of 1Password’s code and data found four choices that make its 26% clean-fix rate a misleading guide to ordinary patching work.¹

• The sample was selected for difficult fixes. The authors chose six vulnerabilities because their fixes were complex. Clean-fix rates ranged from 3% to 60% across those bugs, so the average depends heavily on which vulnerabilities made the list.²
• Two prompts tell agents to apply the wrong fix. Those prompts account for 22% of the data. Combining them with ordinary repair attempts makes the reported rate depend partly on how often the researchers chose to give agents bad advice.
• More than a third of the trials prohibit testing. One evaluation mode prevents agents from building or running code and accounts for 36% of the data. The headline combines those trials with experiments in which agents could test their patches and act on the results.
• The models ran at different reasoning settings. GPT-5.5 ran at medium effort and Opus 4.8 at high. These were the tools’ defaults. Neither model was tested at its highest available setting, and the authors did not measure how increasing effort affected the results.

1Password’s headline also obscures a useful result in its own data. We reanalyzed the patches and recorded test results published with the study, keeping trials where agents could run code and were not instructed to apply the wrong fix. In those trials, 2,634 of 3,067 patches generated by 1Password’s models (86%) blocked the supplied exploit. We excluded runs that the study classified as having consulted the upstream fix. Blocking that exploit does not establish a complete repair, but these results show useful patching capability under reasonable working conditions that the headline fails to convey.

The instructions and grading introduce further problems, several of which Davi Ottenheimer has also highlighted:

• The stopping rule and grading criteria disagree. Agents given a proof-of-concept exploit were instructed to stop once their patch defeated it. The grader then evaluated vulnerable paths that the supplied exploit did not exercise.
• The grading penalizes intended behavior changes. Agents were told to leave existing tests untouched, even though a correct fix can require updating tests to reflect changed behavior. We found that 8% of ActiveMQ verdicts penalized an intended behavior change as a regression.
• The automated grades disagree with human review. Models grading their own patches matched human reviewers on the full five-category outcome in 65.9% of reviewed cases. Agreement was 87.7% for whether the original bug was fixed and 70.5% for whether new bugs were introduced. (Table 24)
• Changing the reviewer changes the result. The two models assigned different outcomes to 36.8% of the same patches. The headline averages their assessments. (Table 20)
• The Linux reference fix contains a vulnerability. The authors found 248 generated patches that repeated an off-by-one error in the upstream fix. The automated grader caught that new vulnerability in only 24 of them. (Section 4.4)
• The Chromium grader accepts incomplete repairs. It marked many patches as clean even though they left a use-after-free vulnerability in a callback. (Section 4.9)

The grading errors can penalize valid fixes and let vulnerable patches pass. Combined with the handpicked sample and deliberately bad instructions, they leave the report without a credible basis for its headline. Defenders should not take 1Password’s headline rate seriously as a measure of AI patching ability.

Developers get one in eight fixes wrong under ideal conditions

Understanding agent failures also requires understanding how often developers submit incomplete fixes. Our security consulting work gives us a detailed record of how developers repair vulnerabilities in their own software. We give clients detailed vulnerability reports, then conduct a “fix review” to check whether their proposed patches fully resolve the issues.

Our records connect each vulnerability to the developer’s first proposed fix and our assessment of whether it worked. They preserve unsuccessful attempts that developers revise before an issue is considered resolved.

We reviewed the first fixes submitted for 2,265 vulnerabilities across 236 Trail of Bits security assessments from 2024 to 2026. The developers maintained the affected software, had detailed reports from our engineers, and knew we would review their patches. Even under those favorable conditions, 283 first fixes failed to fully resolve the reported issue: 12.5%, or one in eight.

Accounting for multiple fixes from the same assessment, the 95% confidence interval is 10.5% to 14.5%. Sometimes we point out a mistake in a client’s patch during an informal conversation, and they correct it before the formal fix review. Those early failures may never appear in the review record, so our data can undercount failed first attempts. We also excluded cases where the available records did not establish whether the fix worked. A direct comparison with agents would require the same tasks and working conditions.

What happened to our patches in real projects

Through Patch the Planet, our joint initiative with OpenAI, Trail of Bits has co-authored hundreds of patches for widely used open-source projects. Agents wrote the patches with engineers directing the work and checking the results. Project maintainers then decided whether to merge, revise, or reject each submission.

How maintainers reviewed Patch the Planet patches

We examined the public review history of every Patch the Planet submission in our dataset that maintainers had merged or closed by September 14, 2026: 186 pull requests. 1Password’s benchmark used six vulnerabilities selected because their fixes were complex.

Maintainers merged 126 of our 186 pull requests, an acceptance rate of 67.7%.³ In 91 of those 126 pull requests (72.2%), maintainers accepted the security fix we originally proposed.

Review outcome	PRs	% of merged PRs
Total merged	126	100%
Accepted with no security-relevant revision observed	91	72.2%
Accepted with security-relevant revision observed	33	26.2%
Indeterminate	2	1.6%

Table 1: Changes requested by maintainers for 126 merged Patch the Planet pull requests. Security-related revisions include repairs to a proposed fix and expansions of its security coverage.

Maintainer acceptance does not establish that every patch is correct.

Maintainers closed the other 60 submissions without merging them. Most were superseded by other work or declined for policy, process, scope, or maintenance reasons. Four were explicitly rejected on technical grounds.

Reason for closure	PRs	% of closed PRs
Total closed without merge	60	100%
Superseded, reimplemented, or re-landed elsewhere	36	60.0%
Policy, process, scope, or maintenance reasons	14	23.3%
Duplicate or convergent with another fix	3	5.0%
Explicitly rejected on technical grounds	4	6.7%
Other or indeterminate	3	5.0%

Table 2: Reasons maintainers closed 60 Patch the Planet pull requests without merging

One of those closed submissions was our freenginx patch.

A maintainer and an agent introduced the same freenginx crash

1Password’s case study examines a Patch the Planet fix for a memory-safety bug in freenginx’s embedded Perl module. An agent wrote our patch under the direction of a Trail of Bits engineer. It left one vulnerable code path open and introduced a new crash during request cleanup. The paper’s criticism of our patch is correct.

The maintainer closed our pull request and committed a separate fix. That fix covered all three vulnerable code paths but introduced the same crash during cleanup. The paper documents the maintainer’s regression too.

Both authors encountered the same trap. The original bug allowed Perl to destroy a callback before freenginx used it.

Both fixes kept the callback alive so freenginx could use it later. But if the request timed out first, freenginx would make the request unusable and then release the callback. Releasing it could run Perl code that still tried to use the request, crashing the worker. Both authors missed a problem their fix could cause later, during cleanup. Catching it required looking beyond the original bug to what happened when a request ended early.

Two authors, one human and one agent, working separately, made the same mistake on the same bug. Readers deciding whether to use agents need to know how their failures compare with those of human developers. Establishing which is more reliable requires measuring both under comparable conditions.

We checked what happened after our patches were merged

We examined about 33,500 subsequent commits in Patch the Planet projects. When a later commit changed a file our patch had modified, we investigated whether it fixed a problem our patch had introduced. For each suspected regression, an agent attempted to demonstrate its impact with a proof of concept. Other agents and our engineers then challenged the findings.

The review found at least ten functional bugs; four build, test, or release automation bugs; and one performance bug. It found no exploitable security vulnerabilities. Two examples illustrate the problems we identified:

• In go-jose, PR #240 fixed a missing-header crash but exposed an existing validation gap, allowing encrypted messages to succeed even when their key length contradicted the declared algorithm. PR #266 added explicit key-length checks before decryption.
• In Noble FROST, PR #250 returned cached round-two results without first checking whether a retry contained the same authenticated transcript. Changed or stale retry data could therefore bypass that check. The maintainer corrected the behavior by validating retries against the original transcript before returning cached results.

We are extending this investigation to every patch we authored, including patches with maintainer contributions. The findings will help us add checks that catch these failures before we submit future patches.

Agent skills for better security patches

We are releasing two agent skills alongside this post: post-patch-validation to help agents test security fixes, and review-walkthrough to help engineers review code changes.

Post-patch-validation is a new skill we wrote to help agents catch incomplete fixes and regressions before submitting patches for review. It was not used in the Patch the Planet work described above.

The skill starts with a vulnerability report and the code before and after the patch. It guides the agent through four tasks:

• Reproduce the original bug. The agent writes a check that must fail on the vulnerable code and pass on the patched version. A test that passes on both revisions cannot demonstrate a fix.
• Test another path to the same failure. The skill requires at least one distinct variant based on the bug’s root cause, such as a different caller or a cleanup path.
• Check for regressions and new vulnerabilities. It compares behavior that should remain unchanged and tests security properties around the modified code. The plan must also include project tests, a sanitizer check, or a bounded fuzzing run.
• Treat broken test runs as inconclusive. A failed build or missing dependency must not be mistaken for evidence that a vulnerability was reproduced.

Failed checks give the agent specific problems to investigate and repair before submitting its patch. The skill saves the tests and results so maintainers can see what was checked.

To try post-patch-validation, install the skill and give your agent the vulnerability report and the vulnerable and patched revisions:

“Use post-patch-validation to validate the patch in HEAD against <vulnerable-commit>, using the vulnerability report in <report-path>.”

Review-walkthrough helps engineers review the patches they are responsible for merging. It turns a branch’s complete diff into an interactive walkthrough that explains the changes in a logical reading order. Review findings appear beside the relevant code, where engineers can inspect them and draft their own comments. The walkthrough can also prepare a GitHub review for submission. Follow the quick start to generate a walkthrough for your branch.

These releases join our other public agent skills for improving security patches:

• variant-analysis helps agents find related defects elsewhere in the codebase.
• property-based-testing helps them test behavior across generated inputs.
• mutation-testing helps them determine whether their tests detect incorrect behavior and identify missing assertions.

We publish these methods so other teams can use them to examine and improve their own patches.

What a useful patching benchmark should measure

A useful patching benchmark should measure whether agents help developers produce correct fixes and how much review those fixes require. The principles in our 2018 guide to evaluating fuzzing research apply here:

1. Choose a sample that matches the research question. Explain how the sample was chosen and which repair work it represents. Difficult cases can expose failure modes. General failure rates require a representative sample.
2. Measure the effects of working conditions. Give agents appropriate tools and instructions. Report model configurations and test how reasoning settings affect results. Report misleading prompts and restricted tool access separately.
3. Make correctness verifiable. Check that patches fix the vulnerability beyond the supplied exploit. Test for security, functional, and performance regressions. Validate grades against expert review and publish the tests, configurations, and results.
4. Show how results vary. Report per-vulnerability outcomes and variation across repeated attempts. Repeating trials on the same bugs cannot establish that those bugs represent everyday patching.
5. Measure what agents contribute to the repair process. Compare developers working with and without agents on comparable tasks and under comparable conditions. Measure initial patch quality and the review and revision needed to reach a correct fix.

We are optimistic about AI’s usefulness to defenders. Through Patch the Planet, we are committing engineering time to fixing vulnerabilities alongside the people who maintain the affected software. We examine failures so we can improve our methods.

We will keep putting agents to work on difficult security problems and making the tools and lessons public. We want other teams to test our conclusions and take these methods further. Our goal is to give maintainers without dedicated security teams the ability to find and fix vulnerabilities that would otherwise go unaddressed.

Fermat famously claimed to have a “truly marvelous proof” of his Last Theorem, but he never wrote it down, insisting the margin of his page was too narrow to contain it. A few centuries later, Anthropic announced a complete formalization of Fermat’s Last Theorem using 13 million lines of Lean code (clearly not what Fermat intended). Luckily, we found a wonderfully cursed Lean bug, shown below, that suggests the proof may have fit the margin after all. The issue affects all stable versions of Lean up to 4.33.1, and the patch is incorporated in v4.34.0-rc1.

The blue checkmarks in the screenshot above would suggest that Lean considers this proof correct. This seems odd given the amount of work Sir Andrew Wiles put into this problem and the vast size of Claude’s proof. So what is going on?

The “proof” clearly doesn’t make any sense and exploits an issue in Lean. We found the issue while using GPT-5.6 to experiment with a new skill for code review. We want to clarify up front that the issue is not a kernel soundness issue, but it happens to nicely fit any discussion of strings, lengths, and substrings.

The issue affects String.Pos.Raw.extract, Lean’s low-level string-slicing function. When asked to extract a one-byte slice at an astronomically large position, Lean’s logical definition returns the empty string. But the compiled native code returns the entire original string. That disagreement is enough to manufacture a contradiction. Lean’s ordinary evaluator “proves” that the tiny slice was empty, while native evaluation “proves” that the very same slice contained “a truly marvelous proof.” Put those together, and Lean concludes that the empty string equals a non-empty string. And once you have a contradiction, you can prove anything, including Fermat’s Last Theorem.

On the bright side, the Lean team was considerably faster than mathematical history. About 90 minutes after we reported the issue, hargoniX opened a fix for the memory-safety problem, and it was merged roughly three hours after filing. The remaining semantic mismatch was fixed by Rob23oba five days after the report, closing the issue. We’d like to give a huge shoutout to hargoniX, Rob23oba, and the Lean team for the fast turnaround.

As a reminder, when dealing with external proofs, follow Lean’s guidance for validating a Lean proof. In our proof-of-concept code above, #print axioms flt shows 'flt' depends on axioms: [propext, Classical.choice, Quot.sound, flt._native.native_decide.ax_1_1]. The extra axiom native_decide adds the compiler to the trusted boundary, and therefore needs to be used with care. Machine-checked proofs will increasingly enable an unprecedented level of trust in mathematical results and critical software. However, more work is needed (e.g., lean4lean and alternative kernel implementations) to ensure that proofs aren’t deemed correct through exploitation of issues in theorem provers.

Fermat’s theorem took 350+ years to prove. If you don’t want to wait that long for your code to be audited, contact us.