Offload Secret Management to AWS Secrets Manager from Amazon EKS

Table of Contents


Secrets in Kubernetes, are Base 64 encoded. As such, its trivial for anyone with access to the secret objects, to decode them & fetch the secret value.

In Amazon EKS, the managed Kubernetes control plane runs on EC2 instances with EBS volumes that are encrypted at rest. So when your secrets are created in etcd & saved to disk, they’re encrypted at rest. However, within the cluster, they’re still essentially plaintext.

A better way to keep secrets “secret”, would be to have them encrypted, preferably with a key of your choosing, until they’re needed in a workload pod. To achieve this, you can store your secrets in either AWS Secrets Manager or SSM Parameter Store (as SecureString), instead of within the EKS cluster.

This article describes how to consume secrets from Secrets Manager & Parameter Store in EKS pods.

How It Works

The Kubernetes community has created a CSI driver to facilitate the use of secrets from external secret stores like AWS Secrets Manager & HashiCorp Vault, in Kubernetes pods.

The Secrets Store CSI Driver allows Kubernetes to mount multiple secrets, keys, & certs stored in enterprise-grade external secret stores into Kubernetes pods as a volume.

β€” Secrets Store CSI Driver

Every secret store has a “provider” for the CSI driver, which is how you use a particular secret store with the driver. To consume secrets from AWS Secrets Manager & parameters from AWS Systems Manager Parameter Store, as mounted volumes in EKS pods, you can use the AWS Secrets and Configuration Provider (ASCP) for Kubernetes Secrets Store CSI Driver.

Install ASCP Helm Chart

To get started, install the ASCP Helm chart in your cluster:

helm repo add eks

helm install aws-secrets-provider \
eks/csi-secrets-store-provider-aws \
--namespace kube-system

Or if you prefer Flux GitOps:

kind: HelmRepository
  name: eks
  namespace: flux-system
  interval: 30m
kind: HelmRelease
  name: aws-secrets-provider
  namespace: kube-system
  releaseName: aws-secrets-provider
      chart: csi-secrets-store-provider-aws
      version: 0.0.3
        kind: HelmRepository
        name: eks
        namespace: flux-system
  interval: 30m

Create Secrets & Parameters & SecretProviderClass

Create the secrets & parameters in AWS & a SecretProviderClass in the namespace where your workload will be deployed:

kind: SecretProviderClass
  name: my-secret-provider
  namespace: my-namespace
  provider: aws
    objects: |
      - objectName: "my-secret"
        objectType: "secretsmanager"
      - objectName: "my-parameter"
        objectType: "ssmparameter"

Grant Permissions to Access Secrets

Create an AWS IAM policy to allow access to the secret & parameter:

    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Action": [
            "Resource": [
            "Effect": "Allow",
            "Action": [
            "Resource": [

Then create a Kubernetes service account with this policy, so the workloads using this service account, can access the secret & parameter:

eksctl create iamserviceaccount \
--name my-service-account --cluster my-cluster \
--attach-policy-arn arn:aws:iam::my-account:policy/my-policy \
--namespace my-namespace --approve --override-existing-serviceaccounts

Deploy Workload

Finally, create a Pod to consume the secret & parameter:

apiVersion: v1
kind: Pod
  name: my-pod
  serviceAccountName: my-service-account
  - name: secrets
      readOnly: true
        secretProviderClass: my-secret-provider
  - name: nginx
    image: nginx
    - name: secrets
      mountPath: /mnt/secrets
      readOnly: true

To view the secrets, exec into the Pod and:

# cd /mnt/secrets/

# ls

# cat my-secret

# cat my-parameter


So there you go: secrets & parameters right in your pods. You can use your own KMS CMKs to encrypt the secrets & parameters & as long as the IAM policy has the right permissions, you can fetch them into your workload pods.

Additionally, if you use the secret rotation feature of AWS Secrets Manager, you can configure the ASCP to reconcile secrets with your pods periodically. See the resources below for more details. Note that this feature is currently in alpha.


Check out the following resources to learn more about secrets in Kubernetes:

About the Author ✍🏻

Harish KM is a Principal DevOps Engineer at QloudX & a top-ranked AWS Ambassador since 2020. πŸ‘¨πŸ»β€πŸ’»

With over a decade of industry experience as everything from a full-stack engineer to a cloud architect, Harish has built many world-class solutions for clients around the world! πŸ‘·πŸ»β€β™‚οΈ

With over 20 certifications in cloud (AWS, Azure, GCP), containers (Kubernetes, Docker) & DevOps (Terraform, Ansible, Jenkins), Harish is an expert in a multitude of technologies. πŸ“š

These days, his focus is on the fascinating world of DevOps & how it can transform the way we do things! πŸš€


Oh hi there πŸ‘‹
It’s nice to meet you.

Sign up to our newsletter to be notified when a new blog post lands.

By subscribing you agree to receive promotional marketing materials and agree with our privacy policy. You may unsubscribe at any time.

Leave a Reply

Your email address will not be published. Required fields are marked *