Login
How to use the AWS Workload Credentials Provider for cross-account secret retrieval and prefetching secrets
If you manage secrets across multiple AWS accounts or need faster secret access for latency-sensitive applications, this post shows you how to meet those requirements using two new features of the AWS Workload Credentials Provider (provider). You will learn how to configure role chaining for cross-account secret retrieval and prefetching of secrets to reduce cold-start latency.
By using role chaining, you can access secrets across AWS accounts through a single provider instance by assuming AWS Identity and Access Management (IAM) roles. Prefetching populates the provider’s in-memory cache with secrets at startup so your application can retrieve secrets without waiting for the first request to trigger a network call at runtime.
What is the AWS Workload Credentials Provider?
AWS Secrets Manager stores and rotates credentials, API keys, and other secrets. The AWS Workload Credentials Provider is a client-side HTTP service that retrieves and caches secrets locally. This reduces latency, improves availability during transient failures, and lowers costs. It supports post-quantum TLS by default, requires no language-specific SDK, and works across Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Lambda. For more details, see the Workload Credentials Provider documentation and GitHub repository.
Security considerations
The Server-Side Requst Forger (SSRF) token prevents unauthorized processes from accessing the provider’s HTTP endpoint. Only applications that can read the token file can retrieve secrets through the provider.
Any identity that can access the provider’s endpoint and SSRF token can retrieve secrets through role chaining. This means users with compute environment access can retrieve cross-account secrets when role assumption is configured. Scope the target role’s permissions to only the secrets required by following the principle of least privilege.
For prefetching, secrets are loaded into the provider’s in-memory cache at startup. Any process that can reach the provider’s localhost endpoint and provide a valid SSRF token can retrieve prefetched secrets from the cache.
Cross-account secret retrieval with role chaining
Organizations might store secrets in a dedicated AWS account, or need to share one secret across applications in different accounts. Until now, cross-account retrieval through the provider required attaching resource-based policies directly to each secret. Some customers prefer IAM role assumption. Before this feature, you had to deploy multiple provider instances with different credentials or build custom credential-switching logic. The provider now supports both approaches: resource-based policies and IAM role assumption. While role assumption is especially useful for cross-account scenarios, it also helps within the same account when secrets are protected by different customer-managed KMS keys.
When you include the roleArn query parameter in a request, the provider uses AWS Security Token Service (AWS STS) AssumeRole to obtain temporary credentials for the specified role and retrieves the secret with those credentials. The provider creates and caches a separate client for each role ARN, so subsequent requests to the same role reuse the existing client. Each role client maintains its own independent cache.
Note: The source account runs the Workload Credentials Provider and your application. The target account contains the secret you want to retrieve. A single provider instance in the source account can assume roles in one or more target accounts.
Prerequisites
- A Workload Credentials Provider built and installed in your environment (see the README for build instructions)
- AWS credentials configured in your compute environment with permission to call
sts:AssumeRoleon the target role ARN- If you also retrieve secrets from the source account through the provider, the credentials need
secretsmanager:GetSecretValueandsecretsmanager:DescribeSecretpermissions for those secrets
- If you also retrieve secrets from the source account through the provider, the credentials need
- A secret in a target AWS account that you want to retrieve
- An IAM role in the target account with a trust policy that allows the provider’s identity to assume it
To build the Workload Credentials Provider
The provider is written in Rust and compiles to a single executable. The following steps are for an RPM-based system such as Amazon Linux 2023:
- Install build dependencies:
sudo yum -y groupinstall "Development Tools" - Install Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source "$HOME/.cargo/env" - Clone the repository and build the provider (use the latest tag available):
git clone --branch <git tag> https://github.com/aws/aws-workload-credentials-provider.git cd aws-workload-credentials-provider cargo build --release
The compiled binary is at target/release/aws-workload-credentials-provider.
To install the Workload Credentials Provider on Amazon EC2
After building the provider, install it as a system service on your EC2 instance and configure access to the SSRF token.
- After configuring your
config.tomlfile (see Configuration options section), run the install script to deploy the provider as a systemd service and generate the SSRF token:cd aws_workload_credentials_provider_common/configuration sudo ./install --config config.toml - Add your application user to the
aws-wcp-tokengroup. This grants your application permission to read the SSRF token file, which is required for all secret retrieval requests:sudo usermod -aG aws-wcp-token <APP_USER>
To install on Amazon ECS, Amazon EKS, or Lambda, see the installation instructions in the GitHub repository.
To verify the installation
- Check that the provider is running:
curl -v -H \ "X-Aws-Parameters-Secrets-Token: $(</var/run/awssmatoken)" \ 'http://localhost:2773/secretsmanager/get?secretId=<YOUR_SECRET_ID>' - You’ll receive a JSON response with the secret value. If you see a connection refused error, check that the provider process is running. If you see a 401 or 403 error, verify the SSRF token file is readable and that the provider’s IAM credentials have
secretsmanager:GetSecretValueandsecretsmanager:DescribeSecretpermissions.
Required permissions
The provider’s base IAM identity requires:
sts:AssumeRoleon the target role ARN
The target role requires:
secretsmanager:GetSecretValuesecretsmanager:DescribeSecret
To configure the target account IAM role
Create an IAM role in the target account with a trust policy that allows the provider’s identity in the source account to assume it. Then attach a policy that grants access to the required secrets.
- Create an IAM role in the target account with a trust policy that allows the provider’s identity in the source account to assume it.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111111111111:role/WCProviderRole" }, "Action": "sts:AssumeRole" } ] } - Attach a policy to this role that grants access to the secret:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": "arn:aws:secretsmanager:us-east-1:222222222222:secret:MyDatabaseSecret"
}
]
}To configure the source account IAM role
Before the provider can assume the role you created in the target account, grant it permission to call sts:AssumeRole. Attach the following policy to the provider’s IAM role in the source account:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole"
}
]
}To retrieve the cross-account secret
Call the Workload Credentials Provider endpoint with the roleArn parameter. The following curl example shows how to retrieve a secret using a different IAM role:
curl -v -H "X-Aws-Parameters-Secrets-Token: $(</var/run/awssmatoken)" 'http://localhost:2773/secretsmanager/get?secretId=MyDatabaseSecret&roleArn=arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole'
The following Python example shows the same operation:
import requests
def get_secret_cross_account():
role_arn = "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole"
url = f"http://localhost:2773/secretsmanager/get?secretId=MyDatabaseSecret&roleArn={role_arn}"
with open('/var/run/awssmatoken') as fp:
token = fp.read()
headers = {
"X-Aws-Parameters-Secrets-Token": token.strip()
}
response = requests.get(url, headers=headers)
if response.status_code == 200:
return response.text
else:
raise Exception(f"Status code {response.status_code} - {response.text}")
You can configure the maximum number of simultaneous assumed roles with the max_roles option in the provider’s TOML configuration file. The default is 20, and the range is 1–20.
Prefetching secrets at startup
By default, the Workload Credentials Provider populates its cache lazily—the first request for a secret triggers a network call to Secrets Manager. Prefetching reduces this cold-start latency by loading secrets at startup.
How prefetching works
You can configure prefetching by adding a [capabilities.secrets_manager.prefetch] section to the provider’s TOML configuration file. You can specify secrets to prefetch in two ways:
- Explicit secrets – List specific secret IDs or ARNs using
[[capabilities.secrets_manager.prefetch.secrets]]entries. - Tag-based discovery – Discover secrets by tag key using
[[capabilities.secrets_manager.prefetch.filter_tags]]entries. The provider callsBatchGetSecretValuewith tag key filters to find and cache all matching secrets.
You can use both methods together. Each entry optionally accepts a role_arn field for cross-account prefetching through role chaining.
Required permissions
The following permissions are required on the IAM role that performs the prefetch, depending on whether the secrets are in the source account or a target account.
secretsmanager:BatchGetSecretValue– Required on the source account role for source-account secrets, or on the target role for cross-account secretssecretsmanager:ListSecrets– Required when using tag-based discovery (filter_tags), on whichever role is performing the discovery
Configuration options
You can tune prefetch behavior with the following options in the [capabilities.secrets_manager.prefetch] section of your TOML configuration file:
- cache_buffer_ratio – The maximum fraction of the cache to fill per caching client during prefetch, in the range 0.1–1.0. The default is 0.8. For example, if your cache holds 100 secrets, a ratio of 0.8 prefetches up to 80, leaving room for 20 on-demand secrets to be cached.
- max_jitter_seconds – The maximum random delay in seconds before starting the prefetch task, in the range 0–10. The default is 0 (no jitter). Use this to prevent fleet-wide synchronized API calls when deploying across many instances.
Example: Prefetch with explicit secrets
The following configuration prefetches two secrets at startup, one from the source account and one from a different account using role chaining:
[capabilities.secrets_manager.prefetch]
secrets = [
{ secret_id = "arn:aws:secretsmanager:us-east-1:111111111111:secret:MySecret-AbCdEf" },
{ secret_id = "cross-account-secret", role_arn = "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole" }
]Example: Prefetch with tag-based discovery
The following configuration discovers and caches all secrets tagged with the Environment key, and all secrets tagged with the Team key in a different account:
[capabilities.secrets_manager.prefetch]
filter_tags = [
{ key = "Environment" },
{ key = "Team", role_arn = "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole" },
]Example: Full configuration
The following example shows a complete provider configuration that combines both features:
[logging]
log_level = "info"
[capabilities.secrets_manager]
http_port = 2773
region = "us-east-1"
[capabilities.secrets_manager.cache]
ttl_seconds = 300
[capabilities.secrets_manager.prefetch]
cache_buffer_ratio = 0.6
max_jitter_seconds = 5
secrets = [
{ secret_id = "arn:aws:secretsmanager:us-east-1:111111111111:secret:MySecret-AbCdEf" },
{ secret_id = "arn:aws:secretsmanager:us-east-1:222222222222:secret:CrossAccount-AbCdEf", role_arn = "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole" },
]
filter_tags = [
{ key = "Environment" },
{ key = "Team", role_arn = "arn:aws:iam::222222222222:role/CrossAccountSecretAccessRole" },
]Start the provider with your configuration file:
./aws-workload-credentials-provider sm start --config config.tomlConclusion
This post showed you how to use role chaining for cross-account secret retrieval and prefetching to reduce cold-start latency. Role chaining simplifies multi-account architectures—a single provider instance can retrieve secrets across accounts using IAM role assumption. Prefetching reduces cold-start latency by populating the provider’s cache before your application makes its first request. Combined, these features let you run the Workload Credentials Provider across multiple accounts with faster secret access.
Further reading
- AWS Workload Credentials Provider documentation
- Access AWS Secrets Manager secrets from a different account
- GitHub repository for source code and README
- AWS Secrets Manager documentation
Submit feedback in the comments below, or contact AWS Support with questions.
