Kubernetes ConfigMaps & Secrets Done Right: Beyond Base64 Traps
· Sourabh G Kulkarni
Why Base64 isn't encryption, volume mounts vs env vars, zero-restart reloads, and External Secrets Operator (ESO) integration with Vault and AWS Secrets Manager.

Every Kubernetes engineer learns early that running echo -n "password" | base64 does not encrypt anything. Base64 is an encoding format designed for safe byte transmission over ASCII channels, not a security boundary. If anyone has read access to your cluster API or namespace, your raw secrets are plain text.
Beyond the Base64 trap, configuration management in Kubernetes breaks production systems in two subtle ways: immutable environment variables requiring full pod restarts and uncontrolled secret sprawl across Git repos.
Here is how production engineering teams configure applications cleanly, securely, and without downtime.
flowchart LR
subgraph External["External Secret Stores (KMS / IAM Guarded)"]
Vault["HashiCorp Vault"]
AWS["AWS Secrets Manager"]
end
subgraph Cluster["Kubernetes Cluster"]
ESO["External Secrets Operator (ESO)"]
Secret["Native k8s Secret"]
subgraph Pod["Application Pod"]
Vol["Mounted Volume (/etc/config)"]
App["App Process (Live Reload)"]
end
end
Vault -->|Fetch via IAM/OIDC| ESO
AWS -->|Fetch via IAM/OIDC| ESO
ESO -->|Auto-Sync & Reconcile| Secret
Secret -->|Projected Volume| Vol
Vol -.->|inotify / polling| App
1. Volume Mounts vs. Environment Variables
Most beginners inject configurations and secrets into containers via env: or envFrom:.
# ❌ The Anti-Pattern: Static Environment Variables
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-credentials
key: password
Why Env Vars Become a Liability in Production
- Crash & Crash Dumps: If an application crashes and generates a core dump or stack trace, environment variables are frequently dumped into stdout/stderr logs.
- Subprocess Leakage: Child processes inherited from the main runtime automatically inherit the parent’s environment.
- No Dynamic Reloading: Environment variables are baked into the container process table at launch (
/proc/1/environ). Updating the Secret in Kubernetes will never update the running app without terminating and recreating the pod.
The Production Standard: Projected Volume Mounts
Mount configurations and secrets as read-only files:
# ✅ The Production Standard: Volume Mount
spec:
containers:
- name: api-server
image: registry.sourabhkulkarni.in/api:v1.2.0
volumeMounts:
- name: app-config
mountPath: /etc/app/config
readOnly: true
volumes:
- name: app-config
configMap:
name: api-config
defaultMode: 0440
When you update the ConfigMap in Kubernetes, the kubelet automatically updates the mounted files inside the container via atomic symlink swaps (usually within 30–60 seconds, governed by the kubelet sync loop). Your app can watch the file path using inotify or library watchers (like Go's fsnotify or Spring Cloud Kubernetes) and refresh config without dropping a single active TCP connection.
2. External Secrets Operator (ESO): Keeping Git Repos Clean
Storing Secrets directly in Git (even in private repositories) is an operational risk. The modern cloud-native pattern uses External Secrets Operator (ESO) to synchronize secrets from HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager directly into native Kubernetes Secrets on demand.
Step 1: The SecretStore Definition
Define the link between your cluster and your cloud KMS:
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: aws-secrets-manager
namespace: production
spec:
provider:
aws:
service: SecretsManager
region: us-east-1
auth:
jwt:
serviceAccountRef:
name: eso-irsa-serviceaccount
Step 2: The ExternalSecret Mapping
Tell Kubernetes which cloud secret to fetch and how frequently to refresh it:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: database-credentials-sync
namespace: production
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: SecretStore
target:
name: db-credentials # The native k8s secret created automatically
creationPolicy: Owner
data:
- secretKey: password
remoteRef:
key: prod/rds/postgres
property: db_password
3. The Production Checklist
| Pattern | Bad Practice | Production Standard |
|---|---|---|
| Storage Format | Plain k8s Secret committed to Git | External Secrets Operator (ESO) syncing from KMS / Vault |
| Injection Method | env / envFrom | Projected read-only volume (mountPath) |
| Config Rotation | Manual pod restarts / rolling restart | File-system watch (inotify) or Reloader operator |
| Permissions | Default 0644 | Strict defaultMode: 0400 or 0440 with non-root user |
By treating configurations as living files and delegating secret storage to IAM-authenticated cloud vaults, you turn Kubernetes configuration from a brittle restart trigger into a resilient, zero-downtime architecture.