If you’ve been running Kubernetes clusters for any meaningful amount of time, you’ve likely encountered a familiar problem: orphaned ConfigMaps and Secrets piling up in your namespaces. These abandoned resources don’t just clutter your cluster—they introduce security risks, complicate troubleshooting, and can even impact cluster performance as your resource count grows.
The reality is that Kubernetes doesn’t automatically clean up ConfigMaps and Secrets when the workloads that reference them are deleted. This gap in Kubernetes’ native garbage collection creates a housekeeping problem that every production cluster eventually faces. In this article, we’ll explore why orphaned resources happen, how to detect them, and most importantly, how to implement sustainable cleanup strategies that prevent them from accumulating in the first place.
Understanding the Orphaned Resource Problem
What Are Orphaned ConfigMaps and Secrets?
Orphaned ConfigMaps and Secrets are configuration resources that no longer have any active references from Pods, Deployments, StatefulSets, or other workload resources in your cluster. They typically become orphaned when:
- Applications are updated and new ConfigMaps are created while old ones remain
- Deployments are deleted but their associated configuration resources aren’t
- Failed rollouts leave behind unused configuration versions
- Development and testing workflows create temporary resources that never get cleaned up
- CI/CD pipelines generate unique ConfigMap names (often with hash suffixes) on each deployment
Why This Matters for Production Clusters
While a few orphaned ConfigMaps might seem harmless, the problem compounds over time and introduces real operational challenges:
Security Risks: Orphaned Secrets can contain outdated credentials, API keys, or certificates that should no longer be accessible. If these aren’t removed, they remain attack vectors for unauthorized access—especially problematic if RBAC policies grant broad read access to Secrets within a namespace.
Cluster Bloat: Kubernetes stores these resources in etcd, your cluster’s backing store. As the number of orphaned resources grows, etcd size increases, potentially impacting cluster performance and backup times. In extreme cases, this can contribute to etcd performance degradation or even hit storage quotas.
Operational Complexity: When troubleshooting issues or reviewing configurations, sifting through dozens of unused ConfigMaps makes it harder to identify which resources are actually in use. This “configuration noise” slows down incident response and increases cognitive load for your team.
Cost Implications: While individual ConfigMaps are small, at scale they contribute to storage costs and can trigger alerts in cost monitoring systems, especially in multi-tenant environments where resource quotas matter.
Detecting Orphaned ConfigMaps and Secrets
Before you can clean up orphaned resources, you need to identify them. Let’s explore both manual detection methods and automated tooling approaches.
Manual Detection with kubectl
The simplest approach uses kubectl to cross-reference ConfigMaps and Secrets against active workload resources. Here’s a basic script to identify potentially orphaned ConfigMaps:
#!/bin/bash
# detect-orphaned-configmaps.sh
# Identifies ConfigMaps not referenced by any active Pods
NAMESPACE=${1:-default}
echo "Checking for orphaned ConfigMaps in namespace: $NAMESPACE"
echo "---"
# Get all ConfigMaps in the namespace
CONFIGMAPS=$(kubectl get configmaps -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}')
for cm in $CONFIGMAPS; do
# Skip kube-root-ca.crt as it's system-managed
if [[ "$cm" == "kube-root-ca.crt" ]]; then
continue
fi
# Check if any Pod references this ConfigMap
REFERENCED=$(kubectl get pods -n $NAMESPACE -o json | \
jq -r --arg cm "$cm" '.items[] |
select(
(.spec.volumes[]?.configMap.name == $cm) or
(.spec.containers[].env[]?.valueFrom.configMapKeyRef.name == $cm) or
(.spec.containers[].envFrom[]?.configMapRef.name == $cm)
) | .metadata.name' | head -1)
if [[ -z "$REFERENCED" ]]; then
echo "Orphaned: $cm"
fi
done
A similar script for Secrets would look like this:
#!/bin/bash
# detect-orphaned-secrets.sh
NAMESPACE=${1:-default}
echo "Checking for orphaned Secrets in namespace: $NAMESPACE"
echo "---"
SECRETS=$(kubectl get secrets -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}')
for secret in $SECRETS; do
# Skip service account tokens and system secrets
SECRET_TYPE=$(kubectl get secret $secret -n $NAMESPACE -o jsonpath='{.type}')
if [[ "$SECRET_TYPE" == "kubernetes.io/service-account-token" ]]; then
continue
fi
# Check if any Pod references this Secret
REFERENCED=$(kubectl get pods -n $NAMESPACE -o json | \
jq -r --arg secret "$secret" '.items[] |
select(
(.spec.volumes[]?.secret.secretName == $secret) or
(.spec.containers[].env[]?.valueFrom.secretKeyRef.name == $secret) or
(.spec.containers[].envFrom[]?.secretRef.name == $secret) or
(.spec.imagePullSecrets[]?.name == $secret)
) | .metadata.name' | head -1)
if [[ -z "$REFERENCED" ]]; then
echo "Orphaned: $secret"
fi
done
Important caveat: These scripts only check currently running Pods. They won’t catch ConfigMaps or Secrets referenced by Deployments, StatefulSets, or DaemonSets that might currently have zero replicas. For production use, you’ll want to check against all workload resource types.
Automated Detection with Specialized Tools
Several open-source tools have emerged to solve this problem more comprehensively:
Kor: Comprehensive Unused Resource Detection
Kor is a purpose-built tool for finding unused resources across your Kubernetes cluster. It checks not just ConfigMaps and Secrets, but also PVCs, Services, and other resource types.
# Install Kor
brew install kor
# Scan for unused ConfigMaps and Secrets
kor all --namespace production --output json
# Check specific resource types
kor configmap --namespace production
kor secret --namespace production --exclude-namespaces kube-system,kube-public
Kor works by analyzing resource relationships and identifying anything without dependent objects. It’s particularly effective because it understands Kubernetes resource hierarchies and checks against Deployments, StatefulSets, and DaemonSets—not just running Pods.
Popeye: Cluster Sanitization Reports
Popeye scans your cluster and generates reports on resource health, including orphaned resources. While broader in scope than just ConfigMap cleanup, it provides valuable context:
# Install Popeye
brew install derailed/popeye/popeye
# Scan cluster
popeye --output json --save
# Focus on specific namespace
popeye --namespace production
Custom Controllers with Kubernetes APIs
For more sophisticated detection, you can build custom controllers using client-go that continuously monitor for orphaned resources. This approach works well when integrated with your existing observability stack:
// Pseudocode example
func detectOrphanedConfigMaps(namespace string) []string {
configMaps := listConfigMaps(namespace)
deployments := listDeployments(namespace)
statefulSets := listStatefulSets(namespace)
daemonSets := listDaemonSets(namespace)
referenced := make(map[string]bool)
// Check all workload types for ConfigMap references
for _, deploy := range deployments {
for _, cm := range getReferencedConfigMaps(deploy) {
referenced[cm] = true
}
}
// ... repeat for other workload types
orphaned := []string{}
for _, cm := range configMaps {
if !referenced[cm.Name] {
orphaned = append(orphaned, cm.Name)
}
}
return orphaned
}
Prevention Strategies: Stop Orphans Before They Start
The best cleanup strategy is prevention. By implementing proper resource management patterns from the beginning, you can minimize orphaned resources in the first place.
Use Owner References for Automatic Cleanup
Kubernetes provides a built-in mechanism for resource lifecycle management through owner references. When properly configured, child resources are automatically deleted when their owner is removed.
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: production
ownerReferences:
- apiVersion: apps/v1
kind: Deployment
name: myapp
uid: d9607e19-f88f-11e6-a518-42010a800195
controller: true
blockOwnerDeletion: true
data:
app.properties: |
database.url=postgres://db:5432
Tools like Helm and Kustomize automatically set owner references, which is one reason GitOps workflows tend to have fewer orphaned resources than imperative deployment approaches.
Implement Consistent Labeling Standards
Labels make it much easier to identify resource relationships and track ownership:
apiVersion: v1
kind: ConfigMap
metadata:
name: api-gateway-config-v2
labels:
app: api-gateway
component: configuration
version: v2
managed-by: argocd
owner: platform-team
data:
config.yaml: |
# configuration here
With consistent labeling, you can easily query for ConfigMaps associated with specific applications:
# Find all ConfigMaps for a specific app
kubectl get configmaps -l app=api-gateway
# Clean up old versions
kubectl delete configmaps -l app=api-gateway,version=v1
Adopt GitOps Practices
GitOps tools like ArgoCD and Flux excel at preventing orphaned resources because they maintain a clear desired state:
- Declarative management: All resources are defined in Git
- Automatic pruning: Tools can detect and remove resources not defined in Git
- Audit trail: Git history shows when and why resources were created or deleted
ArgoCD’s sync policies can automatically prune resources:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp
spec:
syncPolicy:
automated:
prune: true # Remove resources not in Git
selfHeal: true
Use Kustomize ConfigMap Generators with Hashes
Kustomize’s ConfigMap generator feature appends content hashes to ConfigMap names, ensuring that configuration changes trigger new ConfigMaps:
# kustomization.yaml
configMapGenerator:
- name: app-config
files:
- config.properties
generatorOptions:
disableNameSuffixHash: false # Include hash in name
This creates ConfigMaps like app-config-dk9g72hk5f. When you update the configuration, Kustomize creates a new ConfigMap with a different hash. Combined with Kustomize’s --prune flag, old ConfigMaps are automatically removed:
kubectl apply --prune -k ./overlays/production \
-l app=myapp
Set Resource Quotas
While quotas don’t prevent orphans, they create backpressure that forces teams to clean up:
apiVersion: v1
kind: ResourceQuota
metadata:
name: config-quota
namespace: production
spec:
hard:
configmaps: "50"
secrets: "50"
When teams hit quota limits, they’re incentivized to audit and remove unused resources.
Cleanup Strategies for Existing Orphaned Resources
For clusters that already have accumulated orphaned ConfigMaps and Secrets, here are practical cleanup approaches.
One-Time Manual Cleanup
For immediate cleanup, combine detection scripts with kubectl delete:
# Dry run first - review what would be deleted
./detect-orphaned-configmaps.sh production > orphaned-cms.txt
cat orphaned-cms.txt
# Manual review and cleanup
for cm in $(cat orphaned-cms.txt | grep "Orphaned:" | awk '{print $2}'); do
kubectl delete configmap $cm -n production
done
Critical warning: Always do a dry run and manual review first. Some ConfigMaps might be referenced by workloads that aren’t currently running but will scale up later (HPA scaled to zero, CronJobs, etc.).
Scheduled Cleanup with CronJobs
For ongoing maintenance, deploy a Kubernetes CronJob that runs cleanup scripts periodically:
apiVersion: batch/v1
kind: CronJob
metadata:
name: configmap-cleanup
namespace: kube-system
spec:
schedule: "0 2 * * 0" # Weekly at 2 AM Sunday
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
template:
spec:
serviceAccountName: cleanup-sa
containers:
- name: cleanup
image: bitnami/kubectl:latest
command:
- /bin/bash
- -c
- |
# Cleanup script here
echo "Starting ConfigMap cleanup..."
for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
echo "Checking namespace: $ns"
# Get all workload-referenced ConfigMaps
REFERENCED_CMS=$(kubectl get deploy,sts,ds -n $ns -o json | \
jq -r '.items[].spec.template.spec |
[.volumes[]?.configMap.name,
.containers[].env[]?.valueFrom.configMapKeyRef.name,
.containers[].envFrom[]?.configMapRef.name] |
.[] | select(. != null)' | sort -u)
ALL_CMS=$(kubectl get cm -n $ns -o jsonpath='{.items[*].metadata.name}')
for cm in $ALL_CMS; do
if [[ "$cm" == "kube-root-ca.crt" ]]; then
continue
fi
if ! echo "$REFERENCED_CMS" | grep -q "^$cm$"; then
echo "Deleting orphaned ConfigMap: $cm in namespace: $ns"
kubectl delete cm $cm -n $ns
fi
done
done
restartPolicy: OnFailure
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cleanup-sa
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cleanup-role
rules:
- apiGroups: [""]
resources: ["configmaps", "secrets", "namespaces"]
verbs: ["get", "list", "delete"]
- apiGroups: ["apps"]
resources: ["deployments", "statefulsets", "daemonsets"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cleanup-binding
subjects:
- kind: ServiceAccount
name: cleanup-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: cleanup-role
apiGroup: rbac.authorization.k8s.io
Security consideration: This CronJob needs cluster-wide permissions to read workloads and delete ConfigMaps. Review and adjust the RBAC permissions based on your security requirements. Consider limiting to specific namespaces if you don’t need cluster-wide cleanup.
Integration with CI/CD Pipelines
Build cleanup into your deployment workflows. Here’s an example GitLab CI job:
cleanup_old_configs:
stage: post-deploy
image: bitnami/kubectl:latest
script:
- |
# Delete ConfigMaps with old version labels after successful deployment
kubectl delete configmap -n production \
-l app=myapp,version!=v${CI_COMMIT_TAG}
- |
# Keep only the last 3 ConfigMap versions by timestamp
kubectl get configmap -n production \
-l app=myapp \
--sort-by=.metadata.creationTimestamp \
-o name | head -n -3 | xargs -r kubectl delete -n production
only:
- tags
when: on_success
Safe Deletion Practices
When cleaning up ConfigMaps and Secrets, follow these safety guidelines:
- Dry run first: Always review what will be deleted before executing
- Backup before deletion: Export resources to YAML files before removing them
- Check age: Only delete resources older than a certain threshold (e.g., 30 days)
- Exclude system resources: Skip kube-system, kube-public, and other system namespaces
- Monitor for impact: Watch application metrics after cleanup to ensure nothing broke
Example backup and conditional deletion:
# Backup before deletion
kubectl get configmap -n production -o yaml > cm-backup-$(date +%Y%m%d).yaml
# Only delete ConfigMaps older than 30 days
kubectl get configmap -n production -o json | \
jq -r --arg date "$(date -d '30 days ago' -u +%Y-%m-%dT%H:%M:%SZ)" \
'.items[] | select(.metadata.creationTimestamp < $date) | .metadata.name' | \
while read cm; do
echo "Would delete: $cm (created: $(kubectl get cm $cm -n production -o jsonpath='{.metadata.creationTimestamp}'))"
# Uncomment to actually delete:
# kubectl delete configmap $cm -n production
done
<h2 class="wp-block-hea