As organizations scale their cloud infrastructure, maintaining proper lifecycle management of Amazon Machine Images (AMIs) is a critical component of their security and risk management goals. AMIs provide the essential information required to launch Amazon Elastic Compute Cloud (Amazon EC2) instances, however; they present security and compliance challenges if not tracked and managed throughout their lifecycle. This blog post explores how organizations can meet their evolving security and compliance requirements by managing potential vulnerabilities across the AMIs deployed throughout their AWS environment.

At the end of 2024, AWS announced lineage supportfor Amazon EC2, providing source details for your AMIs. With this lineage information, you can trace copied or derived AMIs back to their original source. The source AMI information is available for AMIs that were created using specific API commands like CreateImage, CopyImage, and CreateRestoreImageTask. If the AMI was created using a different API command, the ID and AWS Region of the source AMI don’t appear, which can create visibility gaps that potentially impact security and compliance efforts.

To address these gaps and provide comprehensive AMI governance, organizations need to build additional capabilities to analyze the scope of impact of Common Vulnerabilities and Exposures (CVEs), ensure deployed resources originate from an approved golden image, and respond to audit inquiries that require a clear chain of custody for AMIs. A well-designed solution should also help track and enforce approved AMI creation patterns across all accounts and AWS Regions. The AMI lineage solution described in this post is designed to help you manage your organization’s AMI hierarchy and lifecycle, including tracking AMI origins and usage throughout its AWS environment. By implementing this solution, your security teams can quickly understand the scope of impact when security vulnerabilities are discovered, help ensure compliance with organizational policies, and maintain better visibility into their AMI estate.

The solution in this blog post uses Amazon Neptune, a high-performance graph database, along with native AWS security services to maintain a comprehensive view of AMI relationships and enable proactive security monitoring. With the solution in place, you can enforce controls on AMI sourcing, including validation of marketplace AMIs through service control policies (SCPs), and maintain compliance with organizational and regulatory requirements throughout the AMI lifecycle.

AMI Lineage provides a comprehensive governance solution that uses AWS security services and Neptune to create and maintain a hierarchical graph representation of their AMI relationships. This solution helps security and compliance teams understand the complete history of their AMIs including where they originated from, enforce organizational policies such as requiring all AMIs to be encrypted, and rapidly assess security impacts across their organization.
The solution integrates core AWS services with security and governance capabilities. The core components of the solution in the security tooling account are:

• Neptune: A purpose-built, high-performance graph database securely stores and manages the AMI relationship data.
• AWS Lambdafunctions serve as the processing engine for the solution. They process AMI lifecycle events (such as CreateImage, CopyImage, DeregisterImage), evaluate them against compliance rules, and update the Neptune graph database. The functions are configured with least-privilege AWS Identity and Access Management (IAM) permissions to enhance security.
• Amazon API Gateway provides secure REST endpoints for lineage queries and security assessments. Authentication is handled using a combination of API keys and IAM roles to help ensure that only authorized users and systems can access the data.

From a governance perspective, this solution provides comprehensive AMI origin validation to help ensure AMIs come from approved sources, including the validation of AWS Marketplace AMIs against a list of trusted vendors. Lifecycle management capabilities enforce AMI retention policies and deprecation processes. Compliance monitoring tracks adherence to organizational and regulatory requirements, while security event scope assessment capabilities quickly identify affected resources when security vulnerabilities are discovered. A detailed audit trail maintains a complete history of AMI creation, modification, and usage patterns.

The AMI Lineage solution follows AWS security best practices with a multi-account deployment architecture designed to maximize security while maintaining operational efficiency. The architecture distributes responsibilities across three primary account types: an organization management account, a centralized security tooling account, and multiple member accounts.

This architectural approach helps ensure that sensitive operations and data remain centralized in the security tooling account while enabling distributed monitoring and policy enforcement across the organization. The clear separation of concerns enhances security while maintaining the scalability needed for large-scale AWS deployments.

Figure 1: AMI Lineage solution architecture and workflow

The workflow and architecture shown in figure one includes the following:

1. Policy enforcement: The organization management account is the central point for control. It uses AWS Organizations to enforce SCPs that prevent non-compliant AMI actions across the member accounts.
2. Event capture: When an AMI lifecycle event (like CreateImage or CopyImage) occurs in a member account, a local Amazon EventBridge rule captures it.
3. Centralized processing: The event is securely forwarded from the member account’s EventBridge to the central EventBridge in the security tooling account.
4. Data ingestion and analysis: A Lambda function is triggered in the security tooling account. This function processes the event, analyzes it for compliance, and updates the Neptune graph database with the new AMI relationship data. AWS Security Hub and Amazon GuardDuty in the security tooling account also receive and analyze findings from member accounts.
5. Query and visualization: Security teams query the lineage data through a secure API Gateway endpoint. By doing this, they can to visualize AMI hierarchies, investigate security findings from Security Hub, and assess the scope of impact for a given AMI.

The organization management account serves as the central control point for policy enforcement and organizational oversight. This account hosts SCPs that prevent non-approved AMI usage across the organization and manages organization-wide EventBridge rules that capture AMI events from member accounts. Cross-account trust policies configured in this account enable secure communication between the management account and the security tooling account.

Additionally, the management account establishes Security Hub in delegated administrator mode, designating the security tooling account as the centralized security administrator for the organization. From the security tooling account, Security Hub can be then configured to aggregate all Regions down to one core Region for easier evaluation by security personnel.

The security tooling account acts as the central hub for AMI lineage processing and storage. This account hosts the Neptune graph database cluster with encrypted storage, helping to ensure that AMI relationship data is securely maintained. Lambda functions running in this account process events, handle API requests, and evaluate compliance with least-privilege permissions. API Gateway provides secure REST endpoints for lineage queries and security assessments. Security Hub custom insights and findings are centralized here in the security tooling account as the Security Hub delegated administrator account, along with Amazon Simple Notification Service (Amazon SNS) topics for notifications and alerts. The Amazon Virtual Private Cloud (Amazon VPC) infrastructure supporting these services is also deployed in the security tooling account, providing network-level isolation and security.

The solution enables distributed monitoring and enforcement by deploying lightweight components into each member account across the organization. Each member account includes AWS Config rules for continuous compliance monitoring, cross-account IAM roles to enable secure access from the security tooling account, and local EventBridge rules that forward AMI-related events to the central processing system.

Security and compliance integration extends throughout the solution. IAM manages least-privilege access control and permissions across components. AWS CloudTrail records API activity for audit trails and compliance reporting, while Security Hub centralizes security findings and compliance status across your AMI estate. GuardDuty provides threat detection for AMI-related activities. SCPs enforce organization-wide controls on AMI creation and usage patterns, and AWS Config tracks AMI configuration changes and evaluates compliance rules.

The AMI Lineage solution operates through a continuous monitoring and automated response system that maintains comprehensive visibility into your AMI landscape. When AMI lifecycle events occur in your organization, EventBridge rules capture these activities, including creation, copying, modification, and deregistration events. Lambda functions in the security tooling account are then called upon to process these events with appropriate security controls and update the Neptune graph database in real-time, while CloudTrail logs provide a comprehensive audit trail of AMI-related activities.

The system tracks critical security and compliance metadata that forms the foundation of effective AMI governance. This includes:

• Source AMI information and validation status to help ensure lineage integrity
• Creation method and timestamp data for comprehensive audit trails
• Cross-Region and cross-account relationships to understand the full scope of AMI distribution
• Instance launch history with security context to track usage patterns
• AMI state changes including deprecation and deregistration for lifecycle management
• Compliance status along with policy violations to maintain organizational standards.

Security teams use this comprehensive data through secure API calls to visualize complete AMI hierarchies and relationships, providing clear insight into how AMIs are related across your infrastructure. The compliance of your AMI estate is continuously tracked through a combination of services:

• Detection: AWS Config rules deployed in member accounts check for policy violations (for example, incorrect tags and public permissions).
• Aggregation: These findings, along with vulnerability data from services like Amazon Inspector, are aggregated in AWS Security Hub.
• Correlation: Lambda functions in the security tooling account correlate this information with the lineage data in Neptune. Because of this correlation, you can see not just that an AMI is non-compliant, but also its entire downstream impact. When security events like CVE findings are discovered, teams can quickly assess the scope of impact across their entire AMI estate. The solution monitors AMI usage patterns for security anomalies and enforces governance controls through automated policy checks.

The solution provides robust automated policy enforcement capabilities that operate continuously to maintain security and compliance. The system helps ensure that only approved AMIs with verified lineage history can be used to launch new instances, automatically blocking attempts to use non-compliant images. SCP controls on AMI creation and usage are enforced organization-wide, preventing unauthorized AMI operations before they can impact your environment. When policy violations are detected, the system can trigger automated responses to security events and maintain compliance with organizational standards through real-time enforcement.

Before deploying the AMI Lineage solution, you need to establish the proper security and governance foundation across your organization. Your AWS Organizations management account requires administrative permissions, and your organization must be enabled with all features to support the policies used in this solution. You will also need a dedicated security tooling account to host the solution’s core components, with cross-account IAM roles configured to allow secure access. Finally, essential security services must be configured at the organization level, including Security Hub, CloudTrail organization trails for audit logging, and encryption keys using AWS Key Management Service (AWS KMS) for data protection.

From a technical perspective, ensure you have Python 3.8 or later installed if deploying from a local environment, along with AWS Command Line Interface (AWS CLI) version 2 installed and configured with appropriate security credentials. You’ll also need an Amazon Simple Storage Service (Amazon S3) bucket for deployment artifacts, encrypted using SSE-KMS with a customer-managed key to align with best practices for protecting deployment assets.

The complete AMI Lineage solution is available as open source code in the AWS Samples repository. You can clone the repository and follow the deployment instructions. The repository includes the necessary AWS CloudFormation templates, Lambda functions, and deployment scripts referenced in the following phases.

The deployment process follows a five-phase approach that builds security and compliance capabilities progressively:

1. Security foundations
2. Security controls
3. EventBridge rules
4. Core infrastructure
5. Compliance and monitoring

The first phase establishes the security foundation by configuring AWS Organizations security services. This involves enablingSecurity Hub in the management account and designating the security tooling account as the delegated administrator, enablingnullGuardDuty with the security tooling account configured as thenulldelegated administrator, and enabling an organizational wide CloudTrail trail for audit logging.

# In Organization Management Account: 
# Enable Security Hub and set security tooling account as delegated admin 
aws securityhub enable-organization-admin-account \   
--admin-account-id <security-tooling-account-id> 

# Enable GuardDuty organization with security tooling account as admin   
aws guardduty enable-organization-admin-account \   
--admin-account-id <security-tooling-account-id> 

# Create organization trail with encryption aws cloudtrail create-trail \   
--name ami-lineage-trail \   
--s3-bucket-name <your-secure-bucket> \   
--is-organization-trail \   
--kms-key-id <your-kms-key-id> \   
--enable-log-file-validation

The second phase deploys base security controls through organization-wide SCPs. These policies enforce AMI governance controls by preventing the use of non-approved AMIs and helping to ensure that proper tagging and approval workflows are followed.

# In Organization Management Account: 
# Deploy organization-wide SCPs 
aws organizations create-policy \   
--content file://ami-governance-scp.json \   
--name "AMI-Governance-Controls" \   
--type SERVICE_CONTROL_POLICY 

# Attach to organizational units 
aws organizations attach-policy \   
--policy-id <policy-id> \   
--target-id <ou-id>

The third phase deploys organization-wide EventBridge rules from the management account to capture AMI events across member accounts and forward them to the security tooling account for processing. These rules listen for specific API calls captured by CloudTrail.

An example of the event pattern used to capture CreateImage and CopyImage events looks like this:

{
	"source": ["aws.ec2"],
	"detail-type": ["AWS API Call via CloudTrail"],
	"detail": {
		"eventSource": ["ec2.amazonaws.com"],
		"eventName": [
			"CreateImage",
			"CopyImage",
			"RegisterImage",
			"DeregisterImage"
		]
	}
}

# In Organization Management Account: 
# Deploy organization EventBridge rules 
cd deployment-scripts/organization 
./deploy-organization-resources.sh

The fourth phase focuses on core infrastructure deployment in the security tooling account. This is where the primary processing and storage components are deployed, following security best practices by centralizing sensitive operations in a dedicated account.

# Switch to Security Tooling Account context 
# Deploy Neptune cluster with encryption in security tooling account 
cd deployment-scripts/shared 
./deploy-shared-resources.sh

This deployment script handles multiple components in the security tooling account. The Neptune cluster deployment includes encryption and VPC configuration to help ensure secure storage and access to AMI lineage data. Lambda functions are deployed with security controls and configured with VPC attachment, which allows for secure Neptune access in the VPC, appropriate IAM roles with least-privilege permissions, and environment variables for secure configuration. API Gateway provides secure REST endpoints for external access to AMI lineage data and security assessments.

The fifth phase establishes comprehensive compliance and monitoring capabilities across member accounts. AWS Config rules are deployed to continuously monitor AMI compliance across your organization, while EventBridge rules forward AMI events to the central processing system.

# In each Member Account: 
# Deploy AWS Config Rules and monitoring capabilities 
cd deployment-scripts/child-account   
./deploy-child-account-resources.sh

After deployment, thorough verification helps ensure that security configurations are properly implemented. This includes validating IAM permissions to help ensure least-privilege access, testing security controls to verify SCP enforcement, validating encryption settings acrosscomponents, and confirming that the security tooling account is properly configured as the Security Hub delegated administrator.

When deployed, AMI Lineage provides security operations and compliance monitoring capabilities through its API hosted in the security tooling account and automated monitoring systems. Security teams can query and receive complete AMI security relationships to understand the full context of AMIs in their environment.

When investigating AMIs, the system provides detailed security context including source validation information that confirms:

• Whether AMIs come from marketplace sources or trusted accounts
• Compliance status that shows patch levels and policy adherence
• Vulnerability status with CVE findings and scan results
• Comprehensive lineage data showing the complete chain of AMI relationships and approval history

# Get complete security context for an AMI (API Gateway in Security Tooling Account) 
curl -X GET "https://<api-gateway-id>.execute-api.<region>.amazonaws.com/v1/api/v1/ami/ami-1234567890abcdef0/security-context?include_compliance=true" \  
	-H "x-api-key: <your-api-key>"

For security impact assessments, such as when a new CVE is discovered, the solution provides a powerful scope of impact analysis. By querying the API with a specific finding, security teams can rapidly determine every affected resource across their entire organization that stems from a compromised or vulnerable AMI. Using that information, they can understand the full scope of their exposure and begin remediation. See Security best practices in Amazon API Gateway for helpful considerations while using API Keys.

# Assess for a security finding (Security Tooling Account API) 
curl -X POST "https://<api-gateway-id>.execute-api.<region>.amazonaws.com/v1/api/v1/security-impact" \   
	-H "Content-Type: application/json" \   
	-H "x-api-key: <your-api-key>" \   
	-d '{     "ami_id": 
		"ami-1234567890abcdef0",     
		"finding_type": "CVE",     
		"finding_id": "CVE-2024-XXXX",     
		"severity": "CRITICAL"   
	}'

This analysis returns impact information including:

• Affected AMIs in the lineage chain
• Running instances requiring immediate remediation
• Affected AWS accounts and regions for coordinated response
• Associated auto-scaling groups and launch templates that need updates
• Compliance impact assessment for regulatory reporting
• Detailed remediation steps prioritized by risk level.

Compliance monitoring operates continuously through automated assessment capabilities that evaluate your AMI estate against organizational policies and regulatory requirements. Teams can generate comprehensive compliance reports that show adherence to security standards across their entire infrastructure.

# Generate comprehensive compliance report (Security Tooling Account API) 
curl -X POST "https://<api-gateway-id>.execute-api.<region>.amazonaws.com/v1/api/v1/compliance-assessment" \   
	-H "Content-Type: application/json" \   
	-H "x-api-key: <your-api-key>" \   
	-d '{     
		"rules": [       
    		"required_tags",       
    		"approved_source_validation",       
    		"security_scan_status",       
    		"naming_convention",       
    		"lineage_verification"     
		],     
		"scope": "ORGANIZATION"   
	}'

The solution provides security automation and remediation through configurable automated responses to security events. Security Hub, operating in delegated administrator mode from the security tooling account, can be configured to automatically respond to findings by stopping instances using AMIs with critical vulnerabilities, quarantining instances launched from unapproved sources, and sending immediate notifications for high-severity findings.

Security visualization and reporting capabilities, centralized in the security tooling account, provide real-time dashboards showing:

• Compliance status across the organization
• Scoping visualization for rapid decision-making
• AMI approval workflow status for process monitoring
• Patch compliance metrics for maintaining security posture
• Automated remediation activity logs for audit purposes
• Custom security reports tailored to specific organizational needs.

For security investigations and audit purposes, the solution maintains a queryable audit trail that provides a complete history of AMIs, including creation and modification events, security scanning results and findings, approval workflow history, and compliance status changes over time.

# Query comprehensive audit history (Security Tooling Account API) 
curl -X GET "https://<api-gateway-id>.execute-api.<region>.amazonaws.com/v1/api/v1/ami/ami-1234567890abcdef0/lineage?direction=both&depth=10" \   
	-H "x-api-key: <your-api-key>"

To decommission the AMI Lineage solution, use the following steps to prevent dependency errors. The process is the reverse of the deployment.

1. (Optional) Back up your data. Before you begin, export critical data for your audit and compliance records. This includes generating final compliance reports from the API or creating a final snapshot of the Neptune database (you will be prompted to do this when you delete the cluster).
2. Run cleanup in member accounts. Sign in to each participating member account and run the cleanup script from the deployment files. This removes the local EventBridge rules, AWS Config rules, and cross-account IAM roles.

   # In each Member Account 
   cd deployment-scripts/child-account
   ./cleanup-child-account-resources.sh 
   # Removes Config rules and cross-account roles from each member account

3. Run cleanup in the security tooling account. Sign in to your security tooling account and run the cleanup script. This decommissions the core solution, including the API gateway, Lambda functions, Neptune cluster, and the associated VPC.

   # Clean up security tooling account   
   cd deployment-scripts/shared
   
   ./cleanup-shared-resources.sh 
   # Removes Neptune, Lambda, API Gateway, SNS, and Security Hub components

4. Run cleanup in the organization management account. Sign in to your organization management account to remove the organization-level resources.
   1. Run the cleanup script to remove the organization-wide EventBridge rules.

      # Clean up organization management account
      
      cd deployment-scripts/organization
      
      ./cleanup-organization-resources.sh   
      # Removes SCPs, EventBridge rules, and cross-account trust policies

   2. In the AWS Organizations console, detach and delete the AMI-Governance-Controls SCP.
   3. In the Security Hub and GuardDuty consoles, remove the security tooling account as the delegated administrator.
5. Delete final data and encryption keys. After the solution’s infrastructure is removed, you can delete the remaining assets.
   1. In the security tooling account,empty and delete the S3 bucket that held the deployment artifacts.
   2. In the organization management account,schedule the deletion of the KMS keys you created for encrypting the solution’s data.

In this blog post, we showed you how you can use the AMI Lineage solution to build a comprehensive approach to tracking the complete history of your AMIs from creation to decommissioning. By storing this data in an Amazon Neptune graph database, you can build a hierarchical view of the relationships between your EC2 instances and the AMIs they were launched from. You learned how that data can be used to improve security response and remediation and assist in auditing and compliance activities.

The solution uses AWS Organizations to provide preventative controls to help ensure that only approved AMIs are used and integrates AWS security services like Amazon GuardDuty, AWS Security Hub, and AWS Config to add additional layers of security monitoring and management. Finally, you saw how the solution can be used during a security event or when new CVEs are published, so that you can rapidly discover which systems are affected and automate responses based on those findings.

While this solution provides powerful capabilities, it’s important to consider the operational and cost aspects. The core components, particularly Neptune, have associated costs that will scale with the size of your AMI estate. We recommend implementing cost monitoring and alerts as part of your deployment. Furthermore, because the solution is event-driven, you should plan a one-time backfill process to ingest your organization’s existing AMI history into the graph database. For organizations that require this level of granular control and visibility, these operational considerations are offset by the significant gains in security posture and compliance automation.

AMI Lineage transforms AMI governance from a manual, error-prone process into an automated, comprehensive security capability that scales with your organization’s growth. By implementing this solution, your organization can gain the visibility, control, and automated response capabilities needed to maintain a strong security posture while enabling rapid, secure deployment of infrastructure across its AWS environment.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

AI agents have traditionally faced three core limitations: they can’t retain learned information or operate autonomously beyond short periods, and they require constant supervision. AWS addresses these limitations with frontier agents—a new category of AI that performs complex reasoning, multi-step planning, and autonomous execution for hours or days. Multi-agent collaboration has emerged as a powerful approach that helps tackle complex workflows that require multiple steps and diverse expertise—such as in software development where agents handle code generation, review, and testing; in scientific research where agents collaborate on literature review, experimental design, and data analysis; and in cybersecurity where specialized agents perform reconnaissance, vulnerability analysis, and exploit validation.

In this post, we discuss how we’ve used this technology to deliver automated penetration testing, something that can traditionally take weeks and is resource intensive. We also provide a technical deep-dive into the architecture of the penetration testing component built into AWS Security Agent.

The concept of automated security testing isn’t new—penetration testing tools and vulnerability scanners have existed for decades. However, with recent advancements in large language models (LLMs), frontier agents are designed to reason about application behavior, adapt strategies based on feedback, and understand context in ways that traditional tools can’t. By creating a network of specialized agents, we can address increasingly complex security challenges: one agent maps the attack surface while others analyze business logic flaws, validate findings, and prioritize vulnerabilities based on actual exploitability. The exploitability context comes from the combination of actual exploit attempts by swarm agent workers, independent re-validation by specialized validators, and LLM-driven scoring according to the common vulnerability scoring system (CVSS).

We’ve developed automated penetration testing for the AWS Security Agent. This capability includes a multi-agent penetration testing system that orchestrates specialized security agents to work collaboratively on vulnerability detection. The system begins with multiple types of scanning to establish baseline coverage, then conducts broad reconnaissance using static, predefined tasks to map the application surface and identify initial attack vectors. Building on these findings, our agentic system dynamically generates focused test tasks tailored to the specific application context—reasoning about discovered endpoints, business logic patterns, and potential vulnerability chains to create targeted security tests that adapt based on application responses. By combining these specialized capabilities, the system can tackle complex security scenarios across major risk categories. Beyond single-vulnerability detection, the system performs complex chained attacks—for instance, combining an information disclosure flaw with privilege escalation to access sensitive resources, or chaining insecure direct object references (IDOR) with authentication bypass.

Figure 1: Diagram of the AWS Security Agent penetration testing component.

This section describes the major components of the system. The following subsections cover authentication and initial access, baseline scanning, multi-phased exploration with the specialized agent swarm, and validation with report generation.

The system begins with an intelligent sign-in component that handles authentication across diverse application architectures. This component combines LLM-based reasoning with deterministic mechanisms to locate sign-in pages, attempt provided credentials, and maintain authenticated sessions for subsequent testing phases. The approach adapts to different application structures and target environments automatically and uses a browser tool. The developer can optionally provide a custom sign-in prompt tailored to the target application.

Following authentication, the system initiates comprehensive baseline scanning through parallel execution of specialized scanners. For black-box testing, the network scanner conducts automated web application security testing, generating raw traffic interactions and identifying candidate vulnerable endpoints. In white-box settings, the code scanner additionally performs deep source code analysis when repositories are available, producing descriptive documentation across multiple categories. Additional specialized scanners complement these capabilities to identify vulnerabilities across multiple dimensions and establish initial security coverage.

The system employs two distinct exploration approaches that work in concert. Managed execution operates with predefined static tasks across major risk categories like cross-site scripting, insecure direct object reference, privilege escalation, and so on. This component systematically helps ensure comprehensive coverage by executing curated tasks for each risk type. In the next phase, guided exploration takes a dynamic, intelligence-driven approach. This component ingests discovered endpoints, validated findings, and code analysis documentation to reason about application-specific attack opportunities. It operates in two stages: first generating a contextual penetration testing plan by identifying unexplored resources and potential vulnerability chains, then programmatically managing the execution of these dynamically generated tasks. The guided explorer runs with adaptive tasks that evolve based on application responses and discovered patterns.

Specialized agent swarm
Both exploration approaches dispatch work to specialized swarm worker agents—each configured for specific risk types and equipped with comprehensive penetration testing toolkits including code executors, web fuzzers, NVD vulnerability database search for Common Vulnerabilities and Exposures (CVE) intelligence, and vulnerability-specific tools. These workers execute assigned tasks with timeout management and structured reporting.

When specialized agents identify potential security risks, they generate structured reports containing the vulnerability type, affected endpoints, exploitation evidence, and technical context. However, automated penetration testing faces a critical challenge: LLM agents can produce plausible-sounding findings that require rigorous validation. Candidate findings undergo validation through both deterministic validators and specialized LLM-based agents that attempt active exploitation. We employ assertion-based validation techniques where natural language assertions written by security experts encode deep knowledge about real attack behaviors, requiring explicit, structured proof that’s significantly harder to circumvent than narrow deterministic checks. Validated findings undergo Common Vulnerability Scoring System (CVSS) analysis for severity assessment, then are synthesized into final reports with validation results, severity scores, and exploitation evidence—designed to deliver actionable, high-confidence vulnerabilities for effective remediation.

To evaluate our system, we performed human evaluation in addition to automatic benchmarking. We conducted analysis on real-world trajectories and created a taxonomy of error patterns. By spotting frequent error patterns, we were able to iterate on our solution. We report results on the CVE Bench public benchmark, which is a collection of vulnerable web applications containing 40 critical-severity CVEs from the National Vulnerability Database used to evaluate AI agents on real-world exploits. Each application includes automatic exploit references, and LLM-based agents attempt to execute attacks that trigger the vulnerabilities.

We measure success through the attack success rate (ASR) metric, defined as the rate of successful exploitation of application vulnerabilities. CVE Bench uses a grader that the agent can query to verify exploit success and provides explicit capture-the-flag (CTF) instructions. We evaluate in three configurations:

1. With CTF instructions and grader checks after each tool call, achieving 92.5% on CVE Bench v2.0 (we note that some challenges involve blind exploitation where the agent cannot verify success without this feedback).
2. Without CTF instructions or grader feedback, achieving 80%—which better reflects real-world conditions where the agent must self-validate through observable outcomes. We also observed that the agent was able to identify some CVEs based on the LLM’s parametric knowledge, as shown in the following bash command where the model explicitly references a CVE by name.
3. Therefore, we ran an additional experiment using an LLM whose knowledge cutoff date predates CVE Bench v1.0 release, achieving 65% ASR.

The following code example shows an LLM agent demonstrating parametric knowledge of CVE-2023-37999 from its training data, then issuing a bash command to check exploitation prerequisites.

# HT Mega 2.2.0 has a known vulnerability – CVE-2023-37999
# It has an unauthenticated privilege escalation via the REST API settings endpoint
# Let's check if registration is enabled
curl -s http://target:9090/wp-login.php?action=register -I | head -10

We’re committed to pushing the frontier of security vulnerability detection by continuously evaluating our agent and staying competitive with newer, more challenging benchmarks.

One challenge for penetration testing is determining the balance between exploitation and exploration. Using a depth-first approach can waste too much compute on specific directions, leading to lower vulnerability coverage under a fixed compute budget. Compare that to breadth-first search, which is unlikely to discover deep vulnerabilities that require testing multiple approaches. Therefore, a balance between the two approaches is needed to maximize coverage for a given compute budget. Our proposed system design aims to include a hybrid approach. A more efficient dynamic solution that generalizes across various vulnerabilities and different web applications remains an open research question.

Another challenge with penetration testing is non-determinism. Because of the underlying LLMs, the output of penetration test runs can vary from one run to another. Having different findings across multiple runs can lead to confusion. One option to mitigate this is to perform multiple runs and consolidate the findings across them.

The multi-agent architecture presented in this post demonstrates how you can use specialized agents that can collaborate to tackle complex penetration testing workflows—from intelligent authentication and baseline scanning through managed and guided exploration phases, culminating in rigorous validation. By orchestrating these specialized components with adaptive task generation and assertion-based validation, the system delivers comprehensive security coverage that evolves based on application-specific context and discovered patterns.

AWS Security Agent is now in public preview, for more information, see Getting Started with AWS Security Agent.

Enterprise customers face an unprecedented security landscape where sophisticated cyber threats use artificial intelligence to identify vulnerabilities, automate attacks, and evade detection at machine speed. Traditional perimeter-based security models are insufficient when adversaries can analyze millions of attack vectors in seconds and exploit zero-day vulnerabilities before patches are available.

The distributed nature of serverless architectures compounds this challenge—while microservices offer agility and scalability, they significantly expand the attack surface where each API endpoint, function invocation, and data store becomes a potential entry point, and a single misconfigured component can provide attackers the foothold needed for lateral movement. Organizations must simultaneously navigate complex regulatory environments where compliance frameworks like GDPR, HIPAA, PCI-DSS, and SOC 2 demand robust security controls and comprehensive audit trails, while the velocity of software development creates tension between security and innovation, requiring architectures that are both comprehensive and automated to enable secure deployment without sacrificing speed.

The challenge is multifaceted:

• Expanded attack surface: Multiple entry points across distributed services requiring protection against distributed denial of service (DDoS) attacks, injection vulnerabilities, and unauthorized access
• Identity and access complexity: Managing authentication and authorization across numerous microservices and service-to-service communications
• Data protection requirements: Encrypting sensitive data in transit and at rest while securely storing and rotating credentials without compromising performance
• Compliance and data protection: Meeting regulatory requirements through comprehensive audit trails and continuous monitoring in distributed environments
• Network isolation challenges: Implementing controlled communication paths without exposing resources to the public internet
• AI-powered threats: Defending against attackers who use AI to automate reconnaissance, adapt attacks in real-time, and identify vulnerabilities at machine speed

The solution lies in defense-in-depth—a layered security approach where multiple independent controls work together to protect your application.

This article demonstrates how to implement a comprehensive AI-powered defense-in-depth security architecture for serverless microservices on Amazon Web Services (AWS). By layering security controls at each tier of your application, this architecture creates a resilient system where no single point of failure compromises your entire infrastructure, designed so that if one layer is compromised, additional controls help limit the impact and contain the incident while incorporating AI and machine learning services throughout to help organizations address and respond to AI-powered threats with AI-powered defenses.

Let’s trace a user request from the public internet through our secured serverless architecture, examining each security layer and the AWS services that protect it. This implementation deploys security controls at seven distinct layers with continuous monitoring and AI-powered threat detection throughout, where each layer provides specific capabilities that work together to create a comprehensive defense-in-depth strategy:

• Layer 1 blocks malicious traffic before it reaches your application
• Layer 2 verifies user identity and enforces access policies
• Layer 3 encrypts communications and manages API access
• Layer 4 isolates resources in private networks
• Layer 5 secures compute execution environments
• Layer 6 protects credentials and sensitive configuration
• Layer 7 encrypts data at rest and controls data access
• Continuous monitoring detects threats across layers using AI-powered analysis

Figure 1: Architecture diagram

Before requests reach your application, they traverse the public internet where attackers launch volumetric DDoS attacks, SQL injection, cross-site scripting (XSS), and other web exploits. AWS observed and mitigated thousands of distributed denial of service (DDoS) attacks in 2024, with one exceeding 2.3 terabits per second.

• DDos protection: AWS Shield provides managed DDoS protection for applications running on AWS and is enabled for customers at no cost. AWS Shield Advanced offers enhanced detection, continuous access to the AWS DDoS Response Team (DRT), cost protection during attacks, and advanced diagnostics for enterprise applications.
• Layer 7 protection: AWS WAF protects against Layer 7 attacks through managed rule groups from AWS and AWS Marketplace sellers that cover OWASP Top 10 vulnerabilities including SQL injection, XSS, and remote file inclusion. Rate-based rules automatically block IPs that exceed request thresholds, protecting against application-layer DDoS and brute force attacks. Geo-blocking capabilities restrict access based on geographic location, while Bot Control uses machine learning to identify and block malicious bots while allowing legitimate traffic.
• AI for security: Amazon GuardDuty uses generative AI to enhance native security services, implementing AI capabilities to improve threat detection, investigation, and response through automated analysis.
• AI-powered enhancement: Organizations can build autonomous AI security agents using Amazon Bedrock to analyze AWS WAF logs, reason through attack data, and automate incident response. These agents detect novel attack patterns that signature-based systems miss, generate natural language summaries of security incidents, automatically recommend AWS WAF rule updates based on emerging threats, correlate attack indicators across distributed services to identify coordinated campaigns, and trigger appropriate remediation actions based on threat context. This helps enable more proactive threat detection and response capabilities, reducing mean time to detection and response.

After requests pass edge protection, you must verify user identity and determine resource access. Traditional username/password authentication is vulnerable to credential stuffing, phishing, and brute force attacks, requiring robust identity management that supports multiple authentication methods and adaptive security responding to risk signals in real time.

Amazon Cognito provides comprehensive identity and access management for web and mobile applications through two components:

• User pools offer a fully managed user directory handling registration, sign-in, multi-factor authentication (MFA), password policies, social identity provider integration, SAML and OpenID Connect federation for enterprise identity providers, and advanced security features including adaptive authentication and compromised credential detection.
• Identity pools grant temporary, limited-privilege AWS credentials to users for secure direct access to AWS services without exposing long-term credentials.

Amazon Cognito adaptive authentication uses machine learning to detect suspicious sign-in attempts by analyzing device fingerprinting, IP address reputation, geographic location anomalies, and sign-in velocity patterns, then allows sign-in, requires additional MFA verification, or blocks attempts based on risk assessment. Compromised credential detection automatically checks credentials against databases of compromised passwords and blocks sign-ins using known compromised credentials. MFA supports both SMS-based and time-based one-time password (TOTP) methods, significantly reducing account takeover risk.

For advanced behavioral analysis, organizations can use Amazon Bedrock to analyze patterns across extended timeframes, detecting account takeover attempts through geographic anomalies, device fingerprint changes, access pattern deviations, and time-of-day anomalies.

An API gateway serves as your application’s entry point. It must handle request routing, throttling, API key management, encryption and it needs to integrate seamlessly with your authentication layer and provide detailed logging for security auditing while maintaining high performance and low latency.

• Amazon API Gateway is a fully managed service for creating, publishing, and securing APIs at scale, providing critical security capabilities including SSL/TLS encryption with AWS Certificate Manager (ACM) to automatically handle certificate provisioning, renewal, and deployment. Request throttling and quota management protects backend services through configurable burst and rate limits with usage quotas per API key or client to prevent abuse, while API key management controls access from partner systems and third-party integrations. Request/response validation uses JSON Schema to validate data before reaching AWS Lambda functions, preventing malformed requests from consuming compute resources while seamless integration with Amazon Cognito validates JSON Web Tokens (JWTs) and enforces authentication requirements before requests reach application logic.
• GuardDuty provides AI-powered intelligent threat detection by analyzing API invocation patterns and identifying suspicious activity including credential exfiltration using machine learning. For advanced analysis, Amazon Bedrock analyzes API Gateway metrics and Amazon CloudWatch logs to identify unusual HTTP 4XX error spikes (for example, 403 Forbidden) that might indicate scanning or probing attempts, geographic distribution anomalies, endpoint access pattern deviations, time-series anomalies in request volume, or suspicious user agent patterns.

Application logic and data must be isolated from direct internet access. Network segmentation is designed to limit lateral movement if a security incident occurs, helping to prevent compromised components from easily accessing sensitive resources.

• Amazon Virtual Private Cloud (Amazon VPC) provides isolated network environments implementing a multi-tier architecture with public subnets for NAT gateways and application load balancers with internet gateway routes, private subnets for Lambda functions and application components accessing the internet through NAT Gateways for outbound connections, and data subnets with the most restrictive access controls. Lambda functions run in private subnets to prevent direct internet access, VPC flow logs capture network traffic for security analysis, security groups provide stateful firewalls following least privilege principles, Network ACLs add stateless subnet-level firewalls with explicit deny rules, and VPC endpoints enable private connectivity to Amazon DynamoDB, AWS Secrets Manager, and Amazon S3 without traffic leaving the AWS network.
• GuardDuty provides AI-powered network threat detection by continuously monitoring VPC Flow Logs, CloudTrail logs, and DNS logs using machine learning to identify unusual network patterns, unauthorized access attempts, compromised instances, and reconnaissance activity, now including generative AI capabilities for automated analysis and natural language security queries.

Lambda functions executing your application code and often requiring access to sensitive resources and credentials must be protected against code injection, unauthorized invocations, and privilege escalation. Additionally, functions must be monitored for unusual behavior that might indicate compromise.

Lambda provides built-in security features including:

• AWS Identity and Access Management (IAM) execution roles that define precise resource and action access following least privilege principles
• Resource-based policies that control which services and accounts can invoke functions to prevent unauthorized invocations
• Environment variable encryption using AWS Key Management Services (AWS KMS) for variables at rest while sensitive data should use Secrets Manager function isolation designed so that each execution runs in isolated environments preventing cross-invocation data access
• VPC integration enabling functions to benefit from network isolation and security group controls
• Runtime security with automatically patched and updated managed runtimes
• Code signing with AWS Signer digitally signing deployment packages for code integrity and cryptographic verification against unauthorized modifications

TheAmazon Q Detector Library describes the detectors used during code reviews to identify security and quality issues in code. Detectors contain rules that are used to identify critical security vulnerabilities like OWASP Top 10 and CWE Top 25 issues, including secrets exposure and package dependency vulnerabilities. They also detect code quality concerns such as IaC best practices and inefficient AWS API usage patterns, helping developers maintain secure and high-quality applications.

Vulnerability management: Amazon Inspector provides automated vulnerability management, continuously scanning Lambda functions for software vulnerabilities and network exposure, using machine learning to prioritize findings and provide detailed remediation guidance.

Applications require access to sensitive credentials including database passwords, API keys, and encryption keys. Hardcoding secrets in code or storing them in environment variables creates security vulnerabilities, requiring secure storage, regular rotation, authorized-only access, and comprehensive auditing for compliance.

• Secrets Manager protects access to applications, services, and IT resources without managing hardware security modules (HSMs). It provides centralized secret storage for database credentials, API keys, and OAuth tokens in an encrypted repository using AWS KMS encryption at rest.
• Automatic secret rotation configures rotation for database credentials, automatically updating both the secret store and target database without application downtime.
• Fine-grained access control uses IAM policies to control which users and services access specific secrets, implementing least-privilege access.
• Audit trails log secret access in AWS CloudTrail for compliance and security investigations. VPC endpoint support is designed so that secret retrieval traffic doesn’t leave the AWS network.
• Lambda integration enables functions to retrieve secrets programmatically at runtime, designed so that secrets aren’t stored in code or configuration files and can be rotated without redeployment.
• GuardDuty provides AI-powered monitoring, detecting anomalous behavior patterns that could indicate credential compromise or unauthorized access.

The data layer stores sensitive business information and customer data requiring protection both at rest and in transit. Data must be encrypted, access tightly controlled, and operations audited, while maintaining resilience against availability attacks and high performance.

Amazon DynamoDB is a fully managed NoSQL database providing built-in security features including:

• Encryption at rest (using AWS-owned, AWS managed, or customer managed KMS keys)
• Encryption in transit (TLS 1.2 or higher)
• Fine-grained access control through IAM policies with item-level and attribute-level permissions
• VPC endpoints for private connectivity
• Point-in-Time Recovery for continuous backups
• Streams for audit trails
• Backup and disaster recovery capabilities
• Global Tables for multi-AWS Region, multi-active replication designed to provide high availability and low-latency global access

GuarDuty and Amazon Bedrock provide AI-powered data protection:

• GuardDuty monitors DynamoDB API activity through CloudTrail logs using machine learning to detect anomalous data access patterns including unusual query volumes, access from unexpected geographic locations, and data exfiltration attempts.
• Amazon Bedrock analyzes DynamoDB Streams and CloudTrail logs to identify suspicious access patterns, correlate anomalies across multiple tables and time periods, generate natural language summaries of data access incidents for security teams, and recommend access control policy adjustments based on actual usage patterns versus configured permissions. This helps transform data protection from reactive monitoring to proactive threat hunting that can detect compromised credentials and insider threats.

Even with comprehensive security controls at every layer, continuous monitoring is essential to detect threats that bypass defenses. Security requires ongoing real-time visibility, intelligent threat detection, and rapid response capabilities rather than one-time implementation.

• GuardDuty protects your AWS accounts, workloads, and data with intelligent threat detection.
• CloudWatch provides comprehensive monitoring and observability, collecting metrics, monitoring log files, setting alarms, and automatically reacting to AWS resource changes.
• CloudTrail provides governance, compliance, and operational auditing by logging all API calls in your AWS account, creating comprehensive audit trails for security analysis and compliance reporting.
• AI-powered enhancement with Amazon Bedrock provides automated threat analysis; generating natural language summaries of GuardDuty findings and CloudWatch logs, pattern recognition identifying coordinated attacks across multiple security signals, incident response recommendations based on your architecture and compliance requirements, security posture assessment with improvement recommendations, and automated response through Lambda and Amazon EventBridge that isolates compromised resources, revokes suspicious credentials, or notifies security teams through Amazon SNS when threats are detected.

Securing serverless microservices presents significant challenges, but as demonstrated, using AWS services alongside AI-powered capabilities creates a resilient defense-in-depth architecture that protects against current and emerging threats while proving that security and agility are not mutually exclusive.

Security is an ongoing process—continuously monitor your environment, regularly review security controls, stay informed about emerging threats and best practices, and treat security as a fundamental architectural principle rather than an afterthought.

Further reading

• Best practices for achieving success with Zero Trust
• AI/ML for security – AWS Prescriptive Guidance
• Automate cloud security vulnerability assessment and alerting using Amazon Bedrock
• Infrastructure protection – Serverless Applications Lens

If you have feedback about this blog post, submit them in the Comments section below. If you have questions about using this solution, start a thread in the EventBridge, GuardDuty, or Security Hub forums, or contact AWS Support.

You can use AWS Directory Service for Microsoft Active Directory as your primary Active Directory Forest for hosting your users’ identities. Your IT teams can continue using existing skills and applications while your organization benefits from the enhanced security, reliability, and scalability of AWS managed services. You can also run AWS Managed Microsoft AD as a resource forest. In this configuration, AWS Managed Microsoft AD serves supported AWS services while users’ identities remain under exclusive control of your organization on a self-managed Active Directory. As your organization grows and scales, so will your AWS Managed Microsoft AD deployments.

In this post, you’ll learn how to use Amazon CloudWatch dashboards to monitor key performance metrics of your AWS Managed Microsoft AD deployment to track and analyze a directory’s performance over time. You can then use that information to determine when and how best to scale directory services for optimal performance.

When you deploy AWS Managed Microsoft AD, the service initially creates two domain controller instances in two separate subnets of the same virtual private cloud (VPC). This architecture economically provides resiliency and high availability with a minimal set of resources. This initial configuration enables every feature that AWS Managed Microsoft AD offers. As your organization grows, its workflows will become larger and more complex, requiring that you scale your directories accordingly. AWS Managed Microsoft AD simplifies and makes the scaling process secure with minimal administrative effort. When it’s time to scale a directory, AWS Managed Microsoft AD offers two options: scale-up or scale-out.

Scale-up—also called upgrading your AWS Managed Microsoft AD—means changing the edition of an AWS Managed Microsoft AD from Standard to Enterprise. Enterprise Edition delivers larger domain controller instances, with higher compute capacity and larger storage for Active Directory objects. When a directory scales up, it retains the same number of domain controller instances that it previously had with larger quotas. Instances are replaced one at a time to minimize disruptions to production workflows.

A few features offered by the service are a better fit for the size and compute power of Enterprise Edition AWS Managed Microsoft AD and so are only available in Enterprise Edition. Consider scaling-up your directory if you encounter any of the following scenarios:

• You plan to replicate your directory across multiple AWS Regions. Multi-Region replication is only available in Enterprise Edition.
• The number of Active Directory objects in the directory will exceed the recommended threshold of 30,000 objects for Standard Edition. Enterprise Edition can accommodate up to 500,000 directory objects.
• You plan to share your directory with more than 25 other AWS accounts. The default directory sharing quota is 25 accounts for Standard Edition and 500 for Enterprise Edition.

Important: Scaling up a directory from Standard to Enterprise is a one-way operation that cannot be reverted and operates at a higher hourly price.

Scale-out means deploying additional domain controllers for your AWS Managed Microsoft AD. You can scale out both Standard or Enterprise directories and can scale out different Regions independently. You don’t need to scale every Region to the same number of domain controller instances. When scale-out takes place, additional domain controller instances with the same compute resources and storage capacity as existing ones are launched in the same subnets.

Because some operations cannot be reverted, it’s important to understand the impact of each scaling operation. It’s preferable to scale out the number of domain controllers first, because you can revert that change if necessary. Consider scaling up first only if you need a feature that’s only available in Enterprise Edition.

Since December 2021, AWS Managed Microsoft AD helps optimize scaling decisions with directory metrics in Amazon CloudWatch. Amazon CloudWatch metrics are a time-ordered set of data-points about performance indicators of a system that you can use to monitor and analyze performance over time. Metrics are stored as a time-series set and each data point has an associated timestamp. By using CloudWatch, you can create alarms based on metrics and visualize and analyze metrics to derive new insights.

To understand the performance of a directory over time, define the key performance metrics based on your workload when you create the directory. Record the initial values of those metrics to create a performance baseline. Periodically revisit and compare data points for the same metrics to understand trends and use of resources over time. Based on the information provided by the performance baseline and periodic follow-ups, you can decide when to scale your directory and what scaling method to use. This process is depicted in Figure 1.

Figure 1: Decision-making process for scaling an Active Directory implementation

Depending on the characteristics of your workload, you might face different resource constraints in your directory system. From an infrastructure perspective, the more commonly demanded resources are:

• Network Interface: Current Bandwidth
• Processor: % Processor Time
• LogicalDisk: % Free Space

From an Active Directory perspective, consider metrics such as:

• NTDS: LDAP Searches/sec
• NTDS: ATQ Estimated Queue Delay

The following table is an example decision matrix based on which resource is constrained.

For example, you can create a CloudWatch alarm that will trigger when Processor: % Processor Time is over 80% for more than 5 minutes. If this alarm triggers often, it could be a signal that domain controller instances are struggling to service the regular volume of user authentication requests. In such a scenario, you might consider scaling-out an additional domain controller to guarantee the service’s SLA. Conversely, if the LogicalDisk: % Free Space drops below 10% and trends downwards, you might consider scaling-up to Enterprise Edition, because it provides a larger capacity for directory objects.

To facilitate tracking and analyzing performance of AWS Managed Microsoft AD over time, you can use Amazon CloudWatch to create a custom dashboard including relevant metrics.

Before you get started, make sure that you have the following prerequisites in place:

• An AWS account
• An AWS Identity and Access Management (IAM) user or role with permissions to perform AWS Directory Service operations and CloudWatch operations
• An Amazon Virtual Private Cloud (Amazon VPC) VPC configured in each Region
• At least two private subnets in the VPC
• An AWS Managed Microsoft AD directory

With the prerequisites in place, you’re ready to create a CloudWatch dashboard to track directory service metrics. For more information, see Getting started with CloudWatch automatic dashboards.

To create a dashboard:

1. Open the AWS Management Console for CloudWatch.
2. In the navigation pane, choose Dashboards, and then choose Create dashboard.
3. In the Create new dashboard dialog box, enter a name for the dashboard and then choose Create dashboard.
4. When the Add widget window appears:
   1. Under Data sources types, select CloudWatch.
   2. Under Data type, select Metrics.
   3. Under Widget type, select Line.
   4. Choose Next.
5. In the Add metric graph window, choose DirectoryService and then select Processor as the Metric category and % Processor Time under Metric name. Select each instance of the metric, represented as the Domain Controller IP, for one Directory ID.
6. Choose Create widget.
   

   Note: if there are multiple directories in the same Region, all instances (domain controllers IPs) will be available for selection. To help ensure effective monitoring and alarms, create a separate dashboard for each directory.

7. Choose the plus sign (+) at the top of the window to add more widgets. Repeat steps 1–6 to add additional widgets for other relevant metrics. In this example the metric categories and names added are:
   • Processor: % Processor Time
   • LogicalDisk: % Free Space
   • Memory: Committed Bytes in Use
   • Database: I/O Database Reads Average Latency
   • Network Interface: Current Bandwidth
   • DNS: Recursive Queries/Sec
8. After adding the desired metrics, choose Save.

Figure 2: CloudWatch dashboard showing directory services metrics

Now that you have a dashboard where you can view metrics, consider setting up CloudWatch alarms to alert you when a metric reaches or goes beyond a specified threshold. For more information, see Create a CloudWatch alarm based on a static threshold and Adding an alarm to a CloudWatch dashboard.

The following are recommended thresholds to monitor when determining the need to scale an AWS Managed Microsoft AD. These are general recommendations based on standard use cases. You might have to adjust these thresholds to make the best scaling decisions for your organization.

• Processor: % Processor Time: Monitor CPU utilization to understand computational demands on your domain controllers. Set CloudWatch alarms at 80% for a period of 5 minutes. Sustained high values indicate potential sizing issues that might require scaling out your directory.
• LogicalDisk: % Free Space: Maintain at least 25% free space on volumes containing Active Directory data for optimal performance. Set CloudWatch alarms to trigger when free space drops below 20%. Low disk space can severely impact directory operations and require implementing cleanup procedures or scaling up the directory.
• Network Interface: Current Bandwidth: Average network utilization should be kept below 50% of available bandwidth during peak operations for optimal directory responsiveness. Set CloudWatch alarms at 70% utilization to allow room for spikes in activity. Consistently high values suggest network constraints that might require scaling out your directory.
• Memory: Committed Bytes in Use: Monitor memory commitment levels to help ensure that your domain controllers have sufficient memory resources for Active Directory operations. This metric tracks the amount of virtual memory that has been committed, indicating the total memory load on your domain controllers. Set CloudWatch alarms at 80% of the commit limit. Sustained high values can lead to excessive paging, significantly degrading directory performance and potentially causing authentication delays.
• Database: I/O Database Reads Average Latency: Maintain average read latencies below 25 milliseconds. Set CloudWatch alarms at a threshold of 50 milliseconds. If read latencies are consistently elevated, consider scaling-out your directory.
• DNS: Recursive Queries/sec: Given the tight integration of Active Directory with DNS, monitor this metric for stability and predictable patterns. Use CloudWatch anomaly detection rather than fixed thresholds to identify unexpected behaviors that could indicate DNS configuration issues or potential security concerns.

Different resources across your architecture might contain references to the IP addresses of the AWS Managed Microsoft AD. After a scale-out operation that deploys additional domain controller instances on a directory, update existing references to maintain full functionality of workloads. References for the directory’s IP addresses can be found (but might not be limited to) the following services:

• Firewall rules
• Amazon Virtual Private Cloud (Amazon VPC) security groups
• Amazon Route 53 Resolver endpoint rules
• DNS conditional forwards
• CloudWatch dashboards

To maintain the full functionality of your workloads after a directory scaling operation, update the following:

• Firewall rules that allow traffic to and from the IP addresses of domain controller instances
• Route53 Resolver endpoint rules and DNS conditional forwarders that forward queries to the directory instances
• CloudWatch dashboards that display metric data about the directory to include dimensions for the new IP addresses

In this post, you created components that generate costs. Clean up these resources when no longer required to avoid additional charges.

• Remove added domain controller’s IP addresses from firewall rules, resolver endpoint rules and DNS conditional forwarders.
• Delete the custom CloudWatch dashboards you don’t plan to keep.
• Scale back existing directories to the previous number of domain controller instances.

In this post, you learned how to monitor directory performance metrics using Amazon CloudWatch. By combining performance baselines, monitoring, and planning, you can make informed decisions about when and how to scale a directory safely and efficiently. By scaling directories in a timely manner, you can optimize efficiency and reduce the risk of outages by having a right-sized directory service to support your organization’s workloads.

Scale out your directory when your Active Directory-aware workflows have grown over time and the solution requires additional domain controller instances to maintain the service SLA. Scale up your directory when you require a feature that’s only available in Enterprise Edition AWS Managed Microsoft AD, such as multi-Region replication or additional storage to accommodate Active Directory objects. By using the flexible scaling capabilities and independent Regional expansion, you can optimize costs while maintaining appropriate service levels.

To learn more about AWS Managed Microsoft AD optimization and monitoring with Amazon CloudWatch, see:

• AWS Managed Microsoft AD best practices
• Determining when to add domain controllers with CloudWatch metrics