
Problems of Secrets in GitOps
Kubernetes stores secrets as base64-encoded strings in manifests, so we shouldn’t store secrets’ manifests in a Git repository. That means we cannot practice GitOps for secrets. Sealed Secrets can solve this problem using encryption! In this article, I will explain how to install Sealed Secrets via Helm.
- Preparation
- Install kubeseal
- Install Sealed Secrets
- Test
- (Optional) Manage Sealed Secrets via ArgoCD
- Wrap up
Preparation
Please install tools below.
Required
- kubernetes[1]
- Helm[1]
[1] Here are my versions of each tool.
$ minikube profile list
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|
| Profile | VM Driver | Runtime | IP | Port | Version | Status | Nodes | Active |
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|
| minikube | docker | docker | 192.168.49.2 | 8443 | v1.26.3 | Running | 1 | * |
|----------|-----------|---------|--------------|------|---------|---------|-------|--------|
$ helm version
version.BuildInfo{Version:"v3.12.0", GitCommit:"c9f554d75773799f72ceef38c51210f1842a1dea", GitTreeState:"clean", GoVersion:"go1.20.4"}
Optional
If you’d like to install Sealed Secrets with GitOps way, please prepare below, too.
- ArgoCD[2]
- GitHub repository for manifests
- Please refer to my previous post and create a GitHub repository for manifests.
- ArgoCD Application that maintains ArgoCD Applications by following App of Apps pattern.
- If you’re not familiar with App of Apps pattern, please check the ArgoCD document or my previous post.
[2] Here are my ArgoCD version.
$ argocd version --short | grep argocd-server
argocd-server: v2.7.4+a33baa3.dirty
Install kubeseal
kubeseal is a client-side utility of Sealed Secrets. The kubeseal utility uses asymmetric crypto to encrypt secrets that only the controller can decrypt. The doc explores in more detail how you craft SealedSecret resources.
brew install kubeseal
Install Sealed Secrets
Add the chart repository
Sealed Secrets’s Helm Charts are maintained on bitnami-labs/sealed-secrets. Also, they are hosted at https://bitnami-labs.github.io/sealed-secrets. Let’s add it to your helm.
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
If you’ve already added it, please update it.
helm repo update sealed-secrets
Check available charts’ versions and Sealed Secrets’s versions
You can check available versions via helm search with –versions option.
helm search repo sealed-secrets/sealed-secrets --versions | head -5
example:
NAME CHART VERSION APP VERSION DESCRIPTION
sealed-secrets/sealed-secrets 2.11.0 v0.23.0 Helm chart for the sealed-secrets controller.
sealed-secrets/sealed-secrets 2.10.0 v0.22.0 Helm chart for the sealed-secrets controller.
sealed-secrets/sealed-secrets 2.9.0 v0.21.0 Helm chart for the sealed-secrets controller.
sealed-secrets/sealed-secrets 2.8.2 v0.20.5 Helm chart for the sealed-secrets controller.
This time, I chose the latest version (2.11.0) of Chart that uses v0.23.0 Sealed Secrets App.
Run helm install
Install Sealed Secrets by helm install !
helm install sealed-secrets -n kube-system --set-string fullnameOverride=sealed-secrets-controller sealed-secrets/sealed-secrets --version 2.11.0
Check resources of Sealed Secrets
Sealed Secrets has many k8s’ objects. Let’s take a look at them.
kubectl get -n kube-system "$(kubectl api-resources --namespaced=true --verbs=list -o name | tr "\n" "," | sed -e 's/,$//')" | grep sealed-secrets
endpoints/sealed-secrets-controller 10.244.0.203:8080 105m
pod/sealed-secrets-controller-854f5fbffc-wnwtb 1/1 Running 0 105m
secret/sealed-secrets-keyvhtqf kubernetes.io/tls 2 25h
serviceaccount/sealed-secrets-controller 0 105m
service/sealed-secrets-controller ClusterIP 10.96.79.26 <none> 8080/TCP 105m
deployment.apps/sealed-secrets-controller 1/1 1 1 105m
replicaset.apps/sealed-secrets-controller-854f5fbffc 1 1 1 105m
endpointslice.discovery.k8s.io/sealed-secrets-controller-sflf7 IPv4 8080 10.244.0.203 105m
rolebinding.rbac.authorization.k8s.io/sealed-secrets-controller-key-admin Role/sealed-secrets-controller-key-admin 105m
rolebinding.rbac.authorization.k8s.io/sealed-secrets-controller-service-proxier Role/sealed-secrets-controller-service-proxier 105m
role.rbac.authorization.k8s.io/sealed-secrets-controller-key-admin 2023-07-30T02:02:10Z
role.rbac.authorization.k8s.io/sealed-secrets-controller-service-proxier 2023-07-30T02:02:10Z
You can see many resources such as Services, Deployments and Secrets. Among them, the controller (sealed-secrets-controller) and the Certificate(sealed-secrets-keyvhtqf) are important for encryption and decryption.
Also, you can see CustomResourceDefinitions by the following commands.
kubectl get customresourcedefinitions.apiextensions.k8s.io | grep sealed
sealedsecrets.bitnami.com 2023-07-29T02:21:06Z
Test
Now, we’re ready to use Sealed Secrets! So, let’s create a secret using Sealed Secrets.
Create a sealed secret file
Create a sealed secret file running the command below.
SECRET_VALUE=bar &&
kubectl create secret generic my-secret \
--namespace=default \
--dry-run=client \
--from-literal=foo=${SECRET_VALUE} -o yaml \
| kubeseal \
--controller-namespace=kube-system \
--format yaml \
> my-sealed-secret.yaml
Please check your my-sealed-secret.yaml.
cat my-sealed-secret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: my-secret
namespace: default
spec:
encryptedData:
foo: AgCGnfkL3TFo9Xue/HLxxJ2UkfPeCZKw2QuhUg2vnp7L2FW+xnrF0ECkxm44UFsvZOwH5iIWl1gcDafNhuFSvlnKwJ+yQLa09cDdY3jMyNgqJnqzI3Z4gIdrvXgsCU1dw9EuzmCCeQ/sBBx4DnKYGalhhVqhsqOLX1pK0NPUqnV9VSgzACtysOeclKQk9kVOQ10EShvccwav5yJUFmwRs9ZdY5Gn7+NB4PfTb9TWEtAmXfGvAYSiFSO1UB1u1N0vZYyF/nBV5SRhGLyzJY3K78Xk7Cw4+gaSNfJiYv7lc2wpfjoKihdl/pWFSjkHWPje4aK/iN31Uq9fKLgN6BOnDHMkIeofedEJYpS36l/B8Brudcqf60IFm2GsqIFH0LRPLk5ixz6YCNMekxwfsEaqtbLkYoXoQMwYVCtTP84NIT9o5jqRXGMvD4pXHFt/mmfSa+T4zmx/oGhdJIIs5Lz3QAeILk+egf854Aq1BChFw+5NXZPvQrI8SIqf45dbA6H9uAZ8fBPoyFxiFcN279rg+xnfxloh/+3er1XQY6NlJIymWls5vwVf4QjsXYryc37hLZw6MzmOuYXFjFjNfQQ4fEhRHU8S4EVFVF/QJfT/Lld5Xw1qT4TBhPmHP4m0GKznk1JJaAdKVJe+/aIiVXoLWijyEMVuvqP6Ud8Ok8PhkCixgWfyw+lVG9Nq4Sd6PdLJjTu9nWs=
template:
metadata:
creationTimestamp: null
name: my-secret
namespace: default
Apply the sealed secret
Create SealedSecret in kubernetes running the following command.
kubectl create -f my-sealed-secret.yaml
Check if SealedSecret has been created.
kubectl get sealedsecrets.bitnami.com my-secret
NAME STATUS SYNCED AGE
my-secret True 9s
Check secret
Let’s check if secret is successfully created by the controller.
kubectl get secret my-secret
NAME TYPE DATA AGE
my-secret Opaque 1 28s
Check if the secret’s value.
kubectl get secret my-secret -o=jsonpath={.data.foo} | base64 -d
bar
You can see the value what you encrypted via kubeseal in the previous step!
NOTE: In production usage, please don’t forget limitting the access to the secret using RBAC.
(Optional) Manage Sealed Secrets via ArgoCD
If you’re managing kubernetes add-on using App of Apps pattern of ArgoCD, please do followings.
Create Application manifest of Sealed Secrets
Store the following in a file called sealed-secrets.yaml.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: sealed-secrets
spec:
destination:
name: ''
namespace: kube-system
server: 'https://kubernetes.default.svc'
source:
path: ''
repoURL: 'https://bitnami-labs.github.io/sealed-secrets'
targetRevision: 2.11.0
chart: sealed-secrets
helm:
parameters:
- name: fullnameOverride
value: sealed-secrets-controller
sources: []
project: default
syncPolicy:
automated:
prune: false
selfHeal: false
Push the manifests to manifests repository
git add sealed-secrets.yaml \
&& git commit -m "Add Application manifest of Sealed Secrets" \
&& git push
Here is my commit on GitHub.
Check if it has been installed on ArgoCD UI.
Please sign in your ArgoCD UI, then check if ArgoCD Image Updater has been installed.

Wrap up
We installed Sealed Secrets via Helm and created a sealed secret using kubeseal. The sealed secret’s manifest doesn’t contain the secret value but encrypted value, so we can manage it using Git!
Thank you for reading!
