Enabling security tooling is the starting point. Making it operational—where findings drive decisions, response times are measurable, and your security posture improves week over week—is where most organizations struggle.

This blog post provides a phased maturity roadmap for organizations that have already enabled AWS Security Hub and Amazon GuardDuty. These two services form the foundation of a cloud-centered security operations capability on AWS. Security Hub provides centralized security posture management and aggregates findings from multiple AWS security services, while GuardDuty provides intelligent threat detection by continuously monitoring for malicious activity and unauthorized behavior. For any production or enterprise AWS environment, having both services enabled across all accounts and AWS Regions is a baseline expectation; not because they’re optional add-ons, but because effective security operations require both the ability to detect threats and the ability to understand your overall security posture. If you haven’t yet enabled them, the Security Hub documentation and GuardDuty documentation provide setup guidance, including multi-account deployment with AWS Organizations.

Customers consistently tell us that while individual AWS security service documentation is thorough, what’s missing is a consolidated operational playbook—one resource that ties the services together into a working security operations practice with clear phases, progression criteria, and an operational cadence. That’s the gap this post fills. Rather than covering how each feature works (the documentation does that well), this post focuses on when and why to use each capability, and how to build the organizational habits that make them effective.

What follows is a six-phase roadmap for moving from these services are active to these services are driving our security operations. Each phase builds on the previous one, and each is designed to deliver tangible, measurable improvement.

Goal: Understand what’s working before changing anything.

Estimated timeline: 1–2 weeks

Move to Phase 1 when: You have a documented current-state assessment covering all the following items.

Before introducing new processes or automation, establish a clear picture of the current environment. This assessment informs every decision that follows.

Actions:

• Findings inventory: Review existing active GuardDuty findings to determine how many there are, the severity distribution, and how old the oldest findings are. A large backlog of untouched HIGH or CRITICAL findings that have been sitting for weeks is a strong signal about where to focus first.
• Security Hub score baseline: Determine your current compliance score against AWS Foundational Security Best Practices (FSBP) and The CIS AWS Foundations Benchmark. Check to see which standards are enabled; if multiple standards are enabled, review for overlapping standards (creating noise) or unused standards.
• Multi-account and multi-Region check: Look to see if GuardDuty is enabled in every account and every Region, or only in Regions with active workloads. Threat actors frequently operate in Regions that organizations don’t actively monitor. Also check to see if Security Hub aggregation is configured with a delegated administrator account or if each account is being managed independently.
• Integration check: Determine if GuardDuty findings are flowing into Security Hub and if Amazon Inspector and Amazon Macie are enabled and feeding findings in. Without integration, Security Hub might be only surfacing its own compliance checks.
• Notification check: See if there’s an Amazon EventBridge rule configured for notifications and if so, how findings are being routed and to whom. Know if notifications are being sent using an Amazon Simple Notification Service (Amazon SNS) topic or a chat channel integration. Without a clear notification and response workflow, findings can accumulate silently in the console with no one looking at them.

Deliverable: A one-page current state assessment that identifies what’s enabled, what’s flowing where, who’s looking at it, and what’s in the existing backlog.

Goal: Make the signal meaningful before asking anyone to act on it.

Estimated timeline: 2–3 weeks

Move to Phase 2 when: Remaining findings represent items requiring real decisions, compliance scores reflect actual posture, and you can articulate why every suppression rule and disabled control exists.

This is the single most important phase. If this step is skipped in favor of jumping straight to automation, the result is automated chaos. Alert fatigue is the primary reason security tooling is ignored, and addressing it first is what makes everything that follows sustainable.

GuardDuty tuning:

• Create suppression rules for known-benign findings. The goal is to suppress activity you’ve already evaluated and accepted—such as expected traffic from corporate egress IPs (based on trusted IP lists), internal tools that trigger DNS-based findings, or internet-facing resources that naturally receive port scanning. The principle: if you’ve investigated a finding and it’s expected, suppress it so your team can focus on what matters.
• Triage every active HIGH and CRITICAL finding into three categories: needs immediate investigation (real threat, not yet reviewed), true positive, already addressed (archive using workflow status), or false positive or expected behavior (create a suppression rule). Every finding must be categorized into one of these three states.
• Review GuardDuty protection plans and enable any that are relevant but not yet active. Organizations that enabled GuardDuty years ago might not have activated protection plans released since then (such as Runtime Monitoring, Malware Protection, RDS Protection, and Lambda Protection). Evaluate each against your workload profile and enable what applies.

Security Hub tuning:

• Disable controls that aren’t relevant to the environment. This is the highest-value quick win. If a service isn’t in use, disable its controls. If a control is addressed by an alternative solution, disable it. A 47% compliance score where half the failures are irrelevant trains teams to ignore the dashboard entirely. See the Security Hub controls reference for the full list.
• Choose a primary standard. AWS Foundational Security Best Practices is a strong default. The CIS AWS Foundations Benchmark adds value when there’s a specific compliance mandate. Avoid enabling PCI DSS or NIST 800-53 standards unless there’s a reporting requirement—they add significant volume without proportional signal for most organizations.
• Configure cross-Region aggregation to the delegated administrator account if not already in place. A single aggregated view eliminates the need to check findings across multiple Regional consoles.
• Use the workflow status field operationally. Findings should progress from NEW to NOTIFIED to RESOLVED or SUPPRESSED. If everything remains in NEW indefinitely, the system carries no operational meaning.

Deliverable: A tuned environment where remaining findings represent items that require real decisions. Compliance scores should now reflect your organization’s actual security posture rather than noise.

Goal: Get the right findings to the right people at the right time.

Estimated timeline: 2–3 weeks

Move to Phase 3 when: CRITICAL and HIGH findings reach the security team within minutes, MEDIUM findings create tracked tickets, and notifications include enriched context. No action is taken until a person or an automation is informed that something needs attention.

Architecture: Security Hub to EventBridge rule to routing logic to destination

Tiered notification strategy:

Key design decisions:

• Route from Security Hub, not individual services. Because findings from GuardDuty, Inspector, Macie, and Security Hub compliance checks all aggregate in Security Hub, build your EventBridge rules there for centralized management.
• Create a fast path for the most dangerous finding types. Certain GuardDuty findings, particularly those involving credential exfiltration, cryptocurrency activity, trojans, and active compromises, warrant a separate, faster routing path that bypasses normal triage. Identify these based on your threat model and the GuardDuty finding types reference.
• Enrich notifications before delivery. A raw JSON finding in a chat channel provides little actionable context. Use an AWS Lambda function to format notifications with the information responders need: account alias, Region, Amazon Resource Name (ARN), finding type, severity, a console deep link, and a plain-language description. The Security Hub CloudWatch Events integration guide describes the event format.

Deliverable: A working notification pipeline where CRITICAL and HIGH findings reach the security team within minutes, MEDIUM findings create tracked work items, and LOW and INFORMATIONAL findings are batched for periodic review.

Goal: For findings where the correct response is deterministic, remove the human from the loop.

Estimated timeline: 3–4 weeks

Move to Phase 4 when: At least 3–5 high-confidence finding types have automated responses deployed with audit trails, and the team has established a process for evaluating new auto-remediation candidates.

The guiding principle: Only auto-remediate when all three conditions are met: the finding is high-confidence, the response is deterministic, and the blast radius of the automated action is limited. Automated remediation must not create the risk of a production outage.

Decision framework:

Common auto-remediation categories:

• Instance isolation for confirmed compromise findings (cryptocurrency mining, malware, and trojans): Replace the security group, snapshot volumes for forensics, and notify.
• Credential revocation for confirmed credential compromise: Attach deny-all policies, revoke sessions, and deactivate access keys as appropriate to the credential type.
• Compliance drift correction for deterministic misconfigurations: Re-enable Amazon Simple Storage Service (Amazon S3) Block Public Access, revoke overly permissive security group rules, and re-enable AWS CloudTrail logging.
• Notification-only escalation for findings that require human judgment before action: Amazon Elastic Block Store (Amazon EBS) encryption gaps (require migration) and access key rotation (requires coordination with the key owner).

For implementation, AWS provides Security Hub Automated Response and Remediation (SHARR), a solution that includes pre-built remediation playbooks deployed as AWS Step Functions workflows triggered by EventBridge. This is a strong starting point—evaluate the provided playbooks, enable the ones that fit, and extend with custom remediations as needed.

Note: For findings that recur because the environment lacks preventive guardrails, the best long-term response is often a service control policy (SCP) that prevents the misconfiguration from occurring in the first place. Phase 5 covers this preventive controls layer.

Deliverable: A library of automated and semi-automated remediation runbooks with full audit trails, and a documented decision framework the team uses to evaluate new auto-remediation candidates.

Goal: Turn security findings management into a sustained organizational practice, not a one-time cleanup.

Estimated timeline: 4–6 weeks to establish, then ongoing

Move to Phase 5 when: The weekly cadence has been running consistently for at least 8 weeks, monthly metrics show positive trends, and the first quarterly review has been completed.

This is where many organizations stall, and it’s the most important phase in the entire roadmap. The technology is working, the notifications are flowing, automated remediations are firing, but there’s no organizational habit built around it. Without this phase, everything you’ve built in Phases 0–3 will gradually decay. Suppression rules will go stale, new team members won’t know the system exists, and findings will start accumulating again. The operational rhythm is what converts a security tooling deployment into a security operations practice.

Attendees: Security team lead, cloud platform team representative, rotating engineering lead from an application team

Why the rotating engineering lead matters: Security findings don’t exist in a vacuum; they’re generated by workloads that engineering teams own. Rotating an engineering representative through this meeting accomplishes three things: it builds security awareness across the organization, ensures findings are routed to people with the context to resolve them, and creates organizational accountability beyond the security team.

Agenda template:

Running the meeting effectively:

• Keep a running document (such as a wiki page or shared document) that captures decisions and action items week over week. This becomes your institutional memory.
• If the compliance score hasn’t moved in over 3 weeks, that’s a signal. Either the assigned work isn’t happening, or the remaining findings are genuinely difficult to address. Both need to be discussed.
• Track action items from previous weeks. A review that generates action items but never follows up on them will lose credibility and attendance quickly.

Define clear escalation paths before they’re needed:

Compile these metrics monthly and review them with security and engineering leadership. The goal is to tell a story about whether the organization’s security posture is improving, stable, or degrading, and why.

Building the dashboard: Use Amazon CloudWatch dashboards for real-time operational visibility or Amazon QuickSight connected to Security Hub findings through Amazon Security Lake for historical trend analysis and executive reporting. The dashboard should be visible to—and regularly viewed by—everyone in the weekly review, not locked in a security team tool.

The quarterly review is a deeper inspection of the system itself; not just the findings, but the machinery processing them.

Quarterly review checklist:

• Suppression rules audit: Review every active suppression rule to determine if the underlying condition is still present and the suppression is still justified. Document the review outcome for each rule.
• Disabled controls audit: Review every disabled Security Hub control. Check that the justification is still valid and if the environment changed (for example, a service that wasn’t in use is now in use).
• Automation audit: Review AWS Identity and Access Management (IAM) roles used by remediation functions and verify least privilege. Review execution logs for any anomalies or failures that weren’t caught.
• New capabilities review: Evaluate newly released GuardDuty protection plans and Security Hub controls from that quarter. AWS releases new detection and compliance capabilities regularly. If you’re not reviewing them quarterly, you’re accumulating blind spots.
• Process effectiveness review: Determine if the weekly meeting is well-attended and if action items are being completed. Make sure SLAs are being met. If attendance, action item completion, and SLA compliance aren’t where they should be, explore structural changes to address the gaps.

Use this rubric to assess the maturity of your operational rhythm itself. Score each dimension 1–3 and use the total to track progress over time.

Scoring (revisit quarterly):

• Beginning: 6–9
• Established: 10–14
• Optimized: 15–18

Deliverable: A documented operational cadence with clear ownership (consider a RACI matrix), metrics dashboards, escalation procedures, and a continuous improvement loop. The cadence should survive team member turnover—if it depends on one person remembering to run it, it’s not yet operational.

Goal: Fill remaining gaps and build toward a comprehensive security operations capability. Estimated timeline: Ongoing. Prioritize based on organizational risk profile and compliance requirements.

• Amazon Inspector integration: Enable Amazon Inspector for Amazon Elastic Compute Cloud (Amazon EC2) instances, Lambda functions, and Amazon Elastic Container Registry (Amazon ECR) container images. Findings flow into Security Hub automatically, adding vulnerability management alongside threat detection and posture management. Prioritize this if you have Amazon EC2 or container workloads without an existing vulnerability scanning solution.
• Amazon Macie: Enable Amazon Macie for S3 buckets containing potentially sensitive data. Particularly important for organizations with compliance requirements around personally identifiable information (PII), protected health information (PHI), or Payment Card Industry (PCI) data. Configure automated sensitive data discovery and route findings to Security Hub.
• Amazon Security Lake: Amazon Security Lake centralizes security-relevant logs in OCSF format for long-term retention, forensic investigation, and threat hunting. This is valuable when you need historical analysis beyond the Security Hub retention window, or when feeding a third-party Security Information and Event Management (SIEM) tool.
• Preventive controls layer: Convert recurring detective findings into preventive policies. Use SCPs to prevent disabling GuardDuty, Security Hub, and CloudTrail, IAM permission boundaries on developer roles, AWS WAF on public endpoints, and AWS Network Firewall for VPC traffic inspection. The pattern is to make recurring misconfigurations impossible to introduce.
• Detective controls expansion: Use AWS IAM Access Analyzer for external access and unused access findings, AWS CloudTrail Lake for long-term queryable audit logs, and AWS Config custom rules for organization-specific compliance checks.
• Incident response readiness: Have incident response playbooks referencing specific GuardDuty finding types, pre-built forensics infrastructure (isolated VPC, forensic AMIs, and pre-configured IAM roles), regular tabletop exercises, and AWS CloudFormation templates to deploy isolation infrastructure on demand. See the AWS Security Incident Response Guide for a comprehensive framework.

In this post, I provided a six-phase roadmap for operationalizing Security Hub and GuardDuty and showed that it isn’t a single project, but a progression. Phase 0 and Phase 1 can typically be completed in 3–5 weeks and deliver immediate clarity. Phases 2 and 3 build the response infrastructure that turns findings into action over the following 5–7 weeks. Phase 4 is what makes everything sustainable and is where you should invest the most attention. And Phase 5 expands the aperture from Security Hub and GuardDuty into a comprehensive security operations capability.

If you walked away from this post and did one thing, run the Phase 0 assessment this week. That single deliverable tells you exactly where to focus next. Use the following self-assessment checklist to identify your current phase, then focus on the next one. A tuned environment with working notifications and a weekly review cadence is dramatically more effective than a fully featured but neglected deployment. Start where you are, reduce the noise, build the habits, and iterate. To learn more, explore the AWS Security Hub User Guide, the Amazon GuardDuty User Guide, and the AWS Security Incident Response Guide. If you’ve implemented a similar operational cadence, or have questions about any phase, share your experience in the comments.

If you have feedback about this post, submit comments in the Comments section below.

Modern web applications require robust security controls to protect user data and application resources. Authentication and authorization are two fundamental pillars of application security that answer critical questions: Who are you? and What are you allowed to do? Implementing these controls correctly can be challenging for developers, especially when building data-intensive applications with frameworks like Streamlit (an open-source Python framework for building interactive web applications) or when requiring fine-grained access control. Key challenges include protecting access to application resources, implementing application identity with multi-factor authentication (MFA), and implementing usage-based controls.

In this post, you will learn how to build fine-grained access controls for a sample Streamlit application using Amazon Cognito for authentication and Amazon Verified Permissions with Cedar policies for authorization. This architecture provides enterprise-grade security with minimal development effort, so you can focus on your application’s core functionality. You will learn how to reduce development time for secure applications, implement enterprise-grade authentication, through proper access management, and scale security with growing user bases.

Security architecture overview

The reference architecture follows a layered security design with four key components; separating identity verification, authorization evaluation, application logic, and enforcement boundaries. By assigning clear responsibilities to each layer, the architecture limits blast radius and ensures that a failure in any single control does not compromise the overall system.

• Authentication layer: Amazon Cognito handles user authentication with secure credential validation and JSON web tokens (JWTs). It provides built-in password policies, account lockout protection, and session management.
• Authorization layer: Verified Permissions uses the Cedar policy engine to evaluate fine-grained access requests based on centrally stored policies.
• Application layer: The Streamlit frontend integrates with both services, managing user sessions and enforcing access controls in the user interface.
• Security boundaries: Multiple layers of security controls protect against unauthorized access, privilege escalation, authentication verification, authorization checks, and input validation.

This separation of concerns enables authentication and authorization to function as complementary security controls, following defense-in-depth principles. Figure 1 illustrates the end-to-end authentication and authorization workflow, showing how a user’s sign-in request flows through Amazon Cognito for identity verification, then through Verified Permissions for Cedar policy-based access decisions, before the application enforces the result.

Figure 1: Solution architecture and workflow

The following workflow demonstrates how the three architecture layers work together: the authentication layer (steps 1–3) handles identity verification using Amazon Cognito, the authorization layer (steps 4–6) evaluates Cedar policies using Verified Permissions, and the application layer (steps 7–8) enforces the decision in Streamlit.

1. The user sends a sign-in request, which is submitted through Streamlit
2. The request is authenticated by Amazon Cognito
3. An access token is sent back to Streamlit
4. An authorization request is sent to Verified Permissions
5. The Cedar policy engine evaluates the request
6. A decision is sent back by the policy engine
7. The instruction to allow or deny is sent back to Streamlit
8. If the instruction is to allow, access is provided

Understanding authorization with Cedar

While authentication establishes user identity, authorization determines what actions users can perform. Verified Permissions provides a scalable authorization service based on Cedar, a policy language specifically designed for fine-grained access control.

Cedar policies follow a structured format that defines who can perform which actions on what resources. Let’s examine the anatomy of a Cedar policy:

permit(
    principal == ?principal,
    action == application::Action::"ViewGrade",
    resource == ?resource
) when {
    principal has role == "Student" &&
    resource.student == principal.entityId
};

Policy components

• Effect: permitor forbid determines whether the policy allows or denies access
• Principal: The entity (user) making the request, represented by ?principal as a variable
• Action: The operation being performed, scoped to your application namespace
• Resource: The target of the action, also represented as a variable
• Conditions: The when clause contains logical expressions that must evaluate to true

Advanced Cedar policy patterns

This section describes commonly used Cedar policy patterns for implementing fine-grained authorization with Amazon Verified Permissions. The examples illustrate how to model ownership, role-based access, hierarchical permissions, and administrative controls in real-world applications

Resource ownership control

This pattern helps ensure that users can only access resources they own:

permit(
    principal == ?principal,
    action == application::Action::"ViewGrade",
    resource == ?resource
) when {
    principal has role == "Student" &&
    resource.student == principal.entityId
};

What it does – This policy allows students to view only their own grades by:

• Checking that the user has the Student role
• Verifying that the grade resource’s student attribute matches the student’s entityId
• Preventing students from accessing other students’ grades while allowing access to their own academic performance

Role-based access with resource type

This pattern grants access based on role and resource type:

permit(
    principal == ?principal,
    action == application::Action::"EditCourse",
    resource == ?resource
) when {
    principal has role == "Faculty" &&
    resource has resourceType == "Course" &&
    resource.instructor == principal.entityId
};

What it does – This policy allows faculty members to edit courses they teach by:

• Verifying the user has the Faculty role
• Confirming the resource is of type Course
• Verifying that the course’s instructor attribute matches the faculty member’s entityId
• Restricting faculty to modify only their own courses, not courses taught by other instructors

Hierarchical authorization

This pattern allows department heads to manage faculty in their department:

permit(
    principal == ?principal,
    action == application::Action::"ManageFaculty",
    resource == ?resource
) when {
    principal has role == "DepartmentHead" &&
    resource has role == "Faculty" &&
    resource.department == principal.department
};

What it does – This policy implements departmental hierarchy controls by:

• Requiring the user to be a DepartmentHead
• Verifying the resource is a faculty member
• Matching the faculty member’s department with the department head’s department
• Preventing department heads from managing faculty in other departments

Administrative override

This pattern provides emergency access with proper justification:

permit(
    principal == ?principal,
    action == ?action,
    resource == ?resource
) when {
    principal has role == "Administrator" &&
    context has emergencyAccess == true &&
    context has justification
};

What it does – This policy provides emergency access capabilities by:

• Allowing administrators to perform any action on any resource
• Requiring an emergency access flag to be set to true
• Requiring a justification for emergency access
• Supporting accountability through required documentation while enabling emergency operations

Cedar policy evaluation flow

Understanding how policies are evaluated helps design effective authorization systems. Figure 2 shows a common evaluation pattern for an academic scenario

Note: A policy match evaluates to the policy’s effect (permit or forbid). Forbid policies take precedence: if any forbid policy matches, access is denied regardless of permit policies.

Figure 2: Policy evaluation process

The policy evaluation process follows these steps:

1. User attempts to access a protected resource
2. Application sends an authorization request to Verified Permissions
3. Verified Permissions retrieves applicable Cedar policies from the policy store
4. The Cedar policy engine evaluates each policy against the request
5. If any forbid policy matches, access is denied immediately
6. If any permit policy matches and no forbid policies match, access is allowed
7. If no policies match, access is denied by default
8. The evaluation result (ALLOW or DENY) is returned to the application
9. Application enforces the authorization decision

Cedar policy language

Cedar is an Amazon open source policy language designed for fine-grained authorization. Every policy defines who (principal) can perform what action on which resource under what conditions, as shown in Figure 3.

Figure 3: Cedar policy definitions

Policy interaction

The following table shows how different policies interact in complex scenarios where multiple policies could apply:

Legend:

• Permit – Policy allows access
• N/A – Policy doesn’t apply
• Override – Emergency admin access

The preceding table shows how each role’s policy applies to different scenarios, with admin access having override capabilities across most situations except for emergency admin access where it’s the primary permit authority. The Override column specifically indicates that the administrator’s emergency access policy can supersede other role-specific policies, but only when the emergencyAccess context flag is explicitly set and a justification is provided. This is not an automatic override.

Policy optimization tips:

• Order conditions by likelihood of success – Place the most frequently true conditions first in your when clause to enable short-circuit evaluation. For example, check role before resource ownership, because role mismatches are caught earlier. See Cedar best practices.
• Use indexed attributes for faster lookups – Use entity attributes that Verified Permissions indexes natively (entityId, role, resource type) as primary conditions. Best practices for designing an authorization model
• Cache policy evaluations when appropriate
• Monitor evaluation metrics and performance

Real-world application: Academic system

Consider an academic system with different user roles and their corresponding permissions:

Student: View own grades

• Policy helps ensure students can only access grade resources where they are listed as the student
• The policy verifies the student’s role and matches the resource owner to the principal’s entity ID

Faculty: Edit course content, manage grades

• Policy allows faculty to edit courses they teach
• Faculty can view and modify grades for students in their courses

Teaching assistant (TA): Grade management and course support

• Policy permits TAs to manage grades for courses they assist with
• Access is limited to specific courses assigned to the TA

Department head: Manage faculty assignments

• Policy allows department heads to manage faculty in their department
• Access is scoped to the department hierarchy

Administrator: System-wide access

• Policy provides emergency access with proper justification
• Administrative actions are logged and audited

Prerequisites

To implement the preceding Academic system application, you need an active AWS account, Python 3.8 or later, basic Streamlit knowledge, and AWS Identity and Access Management (IAM) permissions for Amazon Cognito and Verified Permissions.

Run the sample and extend the solution

1. Download the code base: Start by downloading the code base from the avp streamlit samples repository
2. Set up your development environment: Install the AWS SDK for Python (boto3) and configure your AWS credentials.
   • Install the AWS SDK for Python:

     pip install boto3
     

   • Log in to your AWS account:

     aws login --region $REGION

   • Verify that your AWS Command Line Interface (AWS CLI), Python, and dependencies are correctly configured.

     ./verify-setup.sh

3. Create your AWS resources: Use the AWS Management Console or infrastructure as code (IaC) tools to provision your Amazon Cognito user pool and Verified Permissions policy store.

   ./deploy-demo-environment.sh
   Do you want to start the demo now? (Y/N): Y

   This provisions an Amazon Cognito user pool, a Verified Permissions policy store, and any sample resources needed for the demo.

4. Verify the login screen:
   
   

   Figure 4: Verify login credentials

5. Demo walkthrough and shut down: Interact with the demo and test the policies and features. When you’re ready to exit, press Ctrl+C to shut down and stop.
6. Define your Cedar policies: Start with basic policies and gradually add complexity as you understand the evaluation model.
7. Implement authentication: Integrate Amazon Cognito authentication into your application with proper error handling.
8. Add authorization checks: Implement authorization checks at critical access points in your application. For authentication, implement proper error handling for expired tokens, failed MFA challenges, and account lockouts. Use the Amazon Cognito built-in token refresh flow. For authorization, place Verified Permissions checks at every API endpoint and UI component that accesses protected resources.
9. Test thoroughly: Create test scenarios for each user role and permission combination.
10. Monitor and iterate: Set up AWS CloudTrail logging and Amazon CloudWatch alarms to monitor your security controls and refine them based on real-world usage.

Security best practices

When implementing this architecture, follow these best practices to support security:

• Layer your security controls: Use both authentication and authorization as complementary controls rather than relying on a single mechanism.
• Follow least privilege principles: Grant only the permissions needed for specific user roles. Start with minimal permissions and add more as needed.
• Implement proper session management: Set appropriate token expiration and refresh policies. Amazon Cognito handles much of this automatically, but you should configure timeouts based on your security requirements.
• Validate all inputs: Sanitize user inputs to prevent injection attacks. Don’t rely on client-side validation alone.
• Monitor authentication events: Set up logging and alerts for suspicious activities such as repeated failed login attempts or unusual access patterns.
• Conduct regular security reviews: Periodically audit your policies and security configurations to verify they still meet your requirements and follow current best practices.
• Implement secure error handling: Avoid information disclosure through error messages. Provide helpful feedback to users without revealing system details that could aid attackers.

Conclusion

Implementing proper authentication and authorization is critical for application security. By using Amazon Cognito and Amazon Verified Permissions, you can build robust security controls without complex custom code. Through this approach, you can implement enterprise-grade authentication with minimal effort, define and enforce fine-grained authorization policies, scale your security controls as your application grows, and centrally manage and audit security policies.

To get started with your implementation, create your AWS resources including an Amazon Cognito user pool and Verified Permissions policy store. Define your Cedar policies based on your application’s access requirements. Integrate authentication and authorization checks into your application flow. Test thoroughly with different user roles and access scenarios. Finally, monitor and refine your security controls based on usage patterns.

For additional resources, check out the Amazon Cognito documentation and Amazon Verified Permissions documentation.

If you have feedback about this post, submit comments in the Comments section below.

Amazon Cognito recently introduced high-throughput performance for demanding workloads, customer-managed keys for full control over data encryption at rest, and multi- Region replication for business continuity improvement. These capabilities were made possible through a next-generation storage infrastructure designed for extensibility and scale. To deliver this, we migrated hundreds of millions of user profiles, and you probably didn’t even notice. In this post, we walk through what’s new, the architecture behind it, and how we got here with a zero-downtime migration that kept your applications running.

New capabilities now available on Cognito

The migration to the new infrastructure wasn’t just about maintaining existing functionality—it created the foundation for delivering capabilities that solve customer challenges while positioning Amazon Cognito for continuous improvements.

High-throughput performance: The new architecture supports the higher request volumes and scale requirements of modern applications while maintaining the low latency performance that your applications depend on—able to support tens of millions of users per user pool and thousands of transactions per second (TPS).

Customer-managed keys: Customers can now use their own encryption keys stored in AWS Key Management Service (AWS KMS) for encrypting data at rest. This provides enhanced security control and capabilities, giving customers full ownership over their encryption key lifecycle.

Multi-Region replication: Customers can now synchronize their entire user pool data, including user passwords, attributes, and configurations to another user pool in another Region of their choice. This means that customers can implement business continuity strategies and maintain authentication availability in case of a Regional failover, helping their applications remain accessible to users even during unexpected disruptions.

An architecture for innovation

The new architecture uses a purpose-built storage layer designed for extensibility and scale of identity operations. We anchored the new architecture around a set of design tenets:

• Identity-first design: The storage layer understands user identities. There’s no client-specific business logic and no generalizations beyond identity management; keeping the system focused, portable, and optimized.
• Avoid one-way doors: Deliver value incrementally while keeping architectural choices reversible, so we can evolve as new needs arise.
• Backward compatible: Changes to the underlying infrastructure should never break customers’ applications.

These tenets shaped every architectural decision. The architecture separates into independently deployable domains. Previously, while using Amazon Cloud Directory, the service architecture relied on a single data store to persist all customer information. This provided straightforward data traversal mechanisms but required multi-service coordination to adjust database schema when new features were required. The new architecture uses different datasets, allowing them to evolve independently for faster feature iterations.

Migration with zero-downtime

Migrating users requires extreme precautions and a strategy designed to maintain zero downtime and ensure data integrity at every step. Our approach prioritizes both immediate stability and long-term flexibility through the following measures:

1. Shadow mode validation: We ran customer API requests through both old and new infrastructures simultaneously, comparing response structures, status codes, and behavioral characteristics. The validation was designed so that sensitive information was never exposed in plaintext during comparison. We accounted for known variances—for example, timestamps could differ slightly between systems—so that only meaningful discrepancies surfaced as actionable alerts.
2. Data backfill: Before switching a user pool to the new infrastructure, we performed a bulk backfill of all existing user records from the legacy system into the new storage. The backfill ran alongside live traffic with dual-write capturing any changes made during the backfill window, ensuring no data loss or stale data. Shadow mode served as the validation layer for the backfill; as we addressed more edge cases in data syncing, shadow mode match rates increased, confirming data completeness before we proceeded to the switchover.
3. Dual-write architecture: We implemented a system where all identity operations were simultaneously written to both legacy and new services, with comprehensive validation to ensure consistency. Even when a dual-write to the new infrastructure failed, the operation still succeeded in the legacy system, preserving all customer-initiated requests. This means any dual-write failure was contained as an internal consistency issue and not customer-impacting.
4. Anti–entropy validation: We implemented a data validation and correction system that continuously compared records across old and new infrastructures, detecting and resolving any data divergence. Anti-entropy scans compared user attributes, credential hashes, group memberships, and configurations, among other records. When true discrepancies were found, the system automatically reconciled them using the legacy system as the source of truth. This layer was able to catch edge cases that shadow mode and dual writes alone could not cover.
5. Incremental rollout with rollback capability: We established controlled deployment phases with immediate rollback capabilities. After switching a user pool to the new infrastructure, we continued replicating all writes back to the legacy system, ensuring we can revert any user pool to the legacy infrastructure at any point without data loss. If a rollback was needed during migration, an orchestrator replayed entries in timestamp order, syncing user profiles back to the legacy system.

Lessons learned for infrastructure modernization

This modernization taught us valuable principles that apply to any large-scale infrastructure project, therefore we choose to share these learnings to help you perform similar migrations.

• Customer access patterns drive architecture decisions: Analyzing actual customer access patterns revealed that identity workloads follow predictable patterns, which meant we could adopt a synchronous dual-write approach that balanced completeness with operational simplicity. This principle applies to any domain-specific migration: understand your workload’s actual access patterns before reaching for general-purpose solutions.
• Behavioral preservation requires techniques beyond traditional testing: Ensuring equivalent functionality across old and new systems was straightforward. Preserving identical API behavior was not. Functional tests validate intended behaviors, but we identified scenarios where customers had built applications around specific API behaviors such that a change could have silently broken their applications. For example, concurrent writes to the same user could resolve to different final states between old and new systems where writes all succeed but outcome diverges slightly. Similarly, customers who write an attribute and immediately read it are affected by the consistency window. Subtle timing differences in when updates become visible could cause stale reads. These aren’t functional failures, but behavior under real traffic patterns can vary. Shadow mode verification surfaced edge cases that automated tests alone would have missed. Invest in these techniques early.
• Gradual validation builds confidence that testing alone cannot: Layer multiple independent validation techniques, such as shadow mode, dual writes, and anti-entropy scans—each covering a different access pattern. No single approach will catch everything, and the gaps between them are where production issues hide. Incremental rollout with immediate rollback capability lets you validate each step while maintaining the ability to revert quickly.
• Key principles for your own modernization projects: Invest in purpose-built solutions, design for extensibility, and implement gradual validation. Or use managed services so your infrastructure improves without effort on your part while your applications keep running; helping you focus on your business needs.

Conclusion

In this post, we shared the high-level approach and learnings from the Amazon Cognito infrastructure modernization that create a foundation for modern identity management capabilities. The new Cognito infrastructure is live, delivering capabilities such as customer-managed keys and multi-Region replication. As the migration continues, all Cognito customers will gain access to these capabilities on the same service they rely on today, with no action required.

Ready to modernize your authentication infrastructure? Visit Amazon Cognito to learn more.

If you have feedback about this post, submit comments in the Comments section below.

Reconstructing distributed denial of service (DDoS) attack traffic used to mean combining data from multiple sources after the fact. AWS Shield Advanced attack flow logs change that—they capture traffic metadata during attacks so you can pinpoint sources, verify mitigations, and feed your existing analysis pipelines.

Shield publishes logs to Amazon Simple Storage Service (Amazon S3), Amazon CloudWatch Logs, or Amazon Data Firehose using the same CloudWatch Logs delivery infrastructure as other AWS flow logs, so they fit directly into the monitoring and analysis tools you already use.

In this post, you will learn how Shield Advanced attack flow logs capture metadata during DDoS events, what each field in a flow log entry means, and how to enable and configure flow logging for your protected resources.

A DDoS attack floods an application with traffic, making it unavailable to users. Infrastructure-layer attacks saturate bandwidth and exhaust connection tables—you see packet loss and timeouts.

Shield Advanced is a managed DDoS protection service that detects and mitigates attacks for Amazon CloudFront distributions, Elastic Load Balancing load balancers, Amazon Route 53 hosted zones, AWS Global Accelerator standard accelerators, and Elastic IP (EIP) addresses. See the AWS Shield Advanced documentation for full coverage details. Initially, Shield Advanced will provide infrastructure-layer attack flow logs for EIP protections, with support for additional resource types to follow.

Flow logs help you understand attacks in several ways:

• Reconstruct traffic patterns – Query logs after an attack to analyze volume, source distribution, and protocol mix without relying only on aggregate CloudWatch metrics.
• Identify attack origins – The srccountry and location fields show where traffic originated and which AWS edge location it entered.
• Verify mitigation behavior – The action field records what Shield did with each flow.

Logs go to Amazon S3, CloudWatch Logs, or Data Firehose. You can then query them with Amazon Athena (a serverless query service for analyzing data in Amazon S3), route them to third-party Security Information and Event Management (SIEM) platforms or build CloudWatch Logs Insights queries (an interactive log analysis feature) without deploying new infrastructure.

Log records capture source and destination IP addresses and ports, protocol, packet and byte counts, the action Shield Advanced took, and TCP flags. They also include the AWS ingress location where traffic entered and a two-letter country code for the traffic source when available. Logs are written at 5-minute intervals and are available during an active attack and after it concludes.

The maximum file size is 75 MB. If a file reaches that limit within the 5-minute window, the file will be closed, published, and a new file will start. Flow logs support JSON, plain text, W3C, and Parquet output formats and contain the following fields:

The following steps walk you through creating the CloudWatch Logs delivery resources that connect a Shield Advanced protection to your preferred log destination.

Before configuring flow logs, make sure you have:

• An active Shield Advanced subscription
• At least one existing Shield Advanced protection
• AWS Identity and Access Management (IAM) permissions to create CloudWatch Logs delivery resources (logs:PutDeliverySource, logs:PutDeliveryDestination, logs:CreateDelivery)

Flow logs incur standard CloudWatch Logs vended log charges, and the destination resources (S3 bucket storage, CloudWatch Logs log group storage, or Firehose data processing) incur separate charges. Review the Vended Logs entry on the CloudWatch pricing page and the pricing for your chosen destination service before enabling flow logs on high-traffic resources.

Log delivery requires three objects:

• DeliverySource – Represents the Shield Advanced protection that produces the logs
• DeliveryDestination – Represents where logs should be sent (Amazon S3, CloudWatch Logs, or Amazon Data Firehose)
• Delivery – Connects the source to the destination

This three-object model lets you reuse destinations across multiple sources and manage delivery pipelines independently. For example, you can send logs from multiple Shield protections to the same S3 bucket by creating multiple DeliverySource objects that reference the same DeliveryDestination.

Because Shield Advanced attack flow logs use the CloudWatch Logs delivery infrastructure, you can aggregate them across accounts and Regions just like other vended logs. Deliver directly to a centralized S3 bucket with a cross-account policy, replicate CloudWatch Logs log groups using cross-account cross-Region centralization rules, or stream to a shared Firehose stream using cross-account subscriptions. Explore these options to build a unified view of DDoS attack traffic across your multi-account, multi-Region footprint.

Choose a destination:

• Option A – S3 bucket: Best for long-term storage and Athena queries. See Creating an S3 bucket.
• Option B – CloudWatch Logs log group: Best for real-time monitoring and CloudWatch Logs Insights. See Create a log group in CloudWatch Logs.
• Option C – Data Firehose delivery stream: Best for streaming to third-party tools and SIEMs. See Create a Firehose stream.

The destination resource needs a policy that grants the CloudWatch Logs delivery service write permissions. The policy varies by destination type. For more information, see Logs sent to Amazon S3, Logs sent to CloudWatch Logs, or Logs sent to Firehose.

For Amazon S3 destinations, you have two options:

• Automatic policy creation: If your bucket has no existing resource policy and you have the s3:GetBucketPolicy and s3:PutBucketPolicy permissions, AWS automatically creates the required policy when you create the delivery in step 6. You can skip to step 3.
• Manual policy update: If you need to customize the policy or your organization requires pre-approved policies, create the policy manually by following the instructions for Logs sent to Amazon S3.

Shield Advanced is a global service and uses the us-east-1 AWS Region for management. Run the following command to list your Shield Advanced protections.

aws shield list-protections \
  --region us-east-1

In the output, copy the ProtectionArn value for the protection you want to log.

Run the following command to create the delivery source, replace <protection-arn> with the ProtectionArn value from step 3.

aws logs put-delivery-source \
  --name my-shield-delivery-source \
  --resource-arn <protection-arn> \
  --log-type FLOW_LOGS \
  --region us-east-1

The --resource-arn is the ARN of your Shield Advanced protection—not the protected resource itself. Shield Advanced creates a separate protection object that wraps your resource, and flow logs are generated by that protection layer rather than the underlying resource.

Run the following command to create the delivery destination, replace <resource-arn> with the ARN of the destination resource you created in step 1.

aws logs put-delivery-destination \
  --name my-shield-delivery-destination \
  --output-format plain \
  --delivery-destination-configuration '{"destinationResourceArn":"<resource-arn>"}' \
  --region us-east-1

The --delivery-destination-configuration parameter takes a JSON object with a destinationResourceArn key whose value is the ARN of your S3 bucket, log group, or Firehose stream.

In the output, copy the value of the top-level ARN field—this is the delivery destination ARN (different from the bucket ARN). You will use this in step 6.

Run the following command to connect the delivery source to the delivery destination, replace <delivery-destination-arn> with the delivery destination ARN from step 5.

aws logs create-delivery \
  --delivery-source-name my-shield-delivery-source \
  --delivery-destination-arn <delivery-destination-arn> \
  --region us-east-1

Run the following command to confirm the delivery is active.

aws logs describe-deliveries \
  --region us-east-1

After delivery is active, Shield Advanced publishes flow log records to your destination during DDoS events.

To avoid ongoing charges, delete the resources you created.

1. Delete the delivery:

   aws logs delete-delivery \
     --id <delivery-id> \
     --region us-east-1

2. Delete the delivery source:

   aws logs delete-delivery-source \
     --name my-shield-delivery-source \
     --region us-east-1

3. Delete the delivery destination:

   aws logs delete-delivery-destination \
     --name my-shield-delivery-destination \
     --region us-east-1

4. (Optional) Back up flow log data if you need to retain logs for compliance or analysis.
5. Delete the destination resource. Warning: Deleting the destination resource will permanently delete all flow log data.

   For an S3 bucket:

   aws s3 rb s3://<bucket-name> \
     --force \
     --region <region>

   For a CloudWatch Logs log group:

   aws logs delete-log-group \
     --log-group-name <log-group-name> \
     --region <region>

   For a Firehose stream:

   aws firehose delete-delivery-stream \
     --delivery-stream-name <stream-name> \
     --region <region>

Shield Advanced attack flow logs provide the visibility you need to understand and respond to DDoS attacks effectively. By integrating with your existing observability infrastructure, they deliver actionable insights without requiring new tooling or complex setup. Enable flow logs on your Shield Advanced protections today to gain immediate visibility into attack patterns and strengthen your DDoS defense posture.

• Build an Athena table to query logs in Amazon S3
• Interactively search and analyze your log data in Amazon CloudWatch Logs
• Create CloudWatch alarms on attack patterns

For the full reference about flow log configuration, see the AWS Shield Advanced documentation.

If you have feedback about this post, submit comments in the Comments section below.