Secret

A core characteristic is that data within a Secret is stored as base64-encoded strings. This is a fundamental point of confusion: base64 is an encoding scheme, not an encryption method. It prevents accidental exposure of binary data in YAML/JSON manifests but provides no cryptographic security.

• Purpose: Allows storage of both plain text and binary data (like TLS certificates) in a text-based format.
• Security Implication: The data is easily reversible. Anyone with kubectl get secret -o yaml permissions can decode the contents. Real security relies on other mechanisms like Role-Based Access Control (RBAC) and encryption at rest.

Secrets are primarily consumed by pods in two ways, providing flexible integration with application code.

• As Files: The most common and secure method. The Secret's key-value pairs are mounted into the container's filesystem as individual files. This allows applications to read secrets via standard file I/O operations. Updates to the Secret can be propagated to already-running pods.
• As Environment Variables: Secret values can be injected directly into a container's environment. This is less secure as environment variables may be exposed in logs or via debugging tools, and updates are not reflected in running pods.

Example: A pod spec mounting a database password Secret as a file at /etc/db/password.

Kubernetes defines several Secret type fields, which define the intended use and structure of the secret data. The type is a hint to controllers and tooling.

Key built-in types include:

• Opaque: The default type for arbitrary user-defined data (key-value pairs).
• kubernetes.io/tls: For storing a TLS certificate and its private key. Keys must be named tls.crt and tls.key.
• kubernetes.io/dockerconfigjson: For storing credentials to pull images from a private Docker registry.
• kubernetes.io/service-account-token: Used by the ServiceAccount controller to store a token for API access.

Using the correct type enables integration with system components (e.g., the Ingress controller automatically uses tls type secrets).

Managing Secrets securely requires a defense-in-depth approach beyond the basic object.

• Encryption at Rest: Enable and configure EncryptionConfiguration for the Kubernetes API server. This ensures Secret data is encrypted with a user-provided key before being written to etcd.
• Least-Privilege RBAC: Restrict get, list, and watch permissions on Secrets using RBAC roles. Use namespaces to isolate secrets.
• Avoid Environment Variables: Prefer file mounts for sensitive data to reduce exposure risk.
• External Secret Managers: For high-security environments, integrate with external systems like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault using providers like the External Secrets Operator (ESO). These systems offer robust rotation, auditing, and fine-grained access policies.

Secrets have a defined lifecycle and can be made immutable for enhanced security.

• Creation & Updates: Secrets are created via kubectl or API calls. They can be updated, and depending on how they are consumed, changes may propagate to pods.
• Immutable Secrets: A Secret can be marked as immutable: true. This prevents any changes to its data, offering benefits:
  • Security: Accidental or malicious updates are blocked.
  • Performance: Reduces load on the kubectl-apiserver and etcd by disabling watches for that Secret.
• Deletion: When a Secret is deleted, pods using it will continue to run but cannot restart if they depend on a mounted Secret that no longer exists.

Kubernetes automatically creates and manages a special class of Secrets linked to ServiceAccounts.

• Automated Creation: When a ServiceAccount is created, a controller automatically generates a Secret of type kubernetes.io/service-account-token. This Secret contains:
  • A namespace-scoped API token.
  • The cluster's CA certificate.
  • The namespace.
• Pod Mounting: This Secret is automatically mounted at /var/run/secrets/kubernetes.io/serviceaccount in every pod, unless the pod spec explicitly disables it (automountServiceAccountToken: false).
• Purpose: Provides the pod with a cryptographic identity to authenticate to the Kubernetes API server, enabling in-cluster communication and automation.

By default, Kubernetes stores Secret data as base64-encoded plaintext in etcd. For production security, you must enable Encryption at Rest. This encrypts the Secret data with a configurable key (e.g., from a Key Management Service like AWS KMS, GCP Cloud KMS, or Azure Key Vault) before it is written to etcd. This protects against attackers who gain access to the underlying storage.

Access to Secrets must be strictly controlled using Kubernetes Role-Based Access Control (RBAC). Best practices include:

• Create fine-grained Roles and RoleBindings (namespace-scoped) or ClusterRoles and ClusterRoleBindings (cluster-scoped).
• Grant get and list permissions on Secrets only to service accounts used by pods that legitimately need them.
• Avoid granting create, update, or delete permissions broadly. Use dedicated service accounts for CI/CD systems that manage Secrets.
• Regularly audit RBAC bindings to detect over-permissioned accounts.

While convenient, mounting Secrets as environment variables poses risks:

• Environment variables are visible to all processes in the container and may be logged inadvertently.
• Child processes inherit environment variables.
• Updates to the Secret are not reflected in already-running pods.

Preferred Method: Mount Secrets as volumes. This creates a file for each key, allowing the application to read the current value. Kubernetes automatically updates the mounted files when the Secret is updated, and the pod can detect the change (e.g., via inotify).

Secrets should have short lifespans and be rotated regularly to limit the blast radius of a compromise.

• Implement automated rotation using your external secret manager or CI/CD pipelines.
• Use init containers or sidecars to fetch fresh secrets at pod startup rather than relying on long-lived credentials baked into images.
• For service accounts, use TokenRequest API to project short-lived, audience-bound tokens instead of relying on the default, long-lived service account token mounted in pods.
• Monitor Secret age and alert on secrets older than a defined threshold (e.g., 90 days).

Prevent secrets from being accidentally committed to version control, a common source of breaches.

• Use .gitignore rigorously for files containing credentials.
• Employ pre-commit hooks and secret scanning tools like GitGuardian, TruffleHog, or Gitleaks to detect and block commits containing high-entropy strings or patterns matching API keys, passwords, and tokens.
• In CI/CD pipelines, scan build logs and artifacts for secret leakage.
• Never use real secrets in development or test environments; instead, use dummy values or local secret management solutions.

A Kubernetes API object used to store non-confidential configuration data as key-value pairs. It is the non-sensitive counterpart to a Secret. Data can be consumed by pods as:

• Environment variables
• Configuration files in a volume
• Command-line arguments

Unlike Secrets, ConfigMap data is stored as plaintext. Use it for configuration like application properties, feature flags, or non-sensitive endpoints.

A Kubernetes identity for pods and processes to authenticate to the Kubernetes API server. Each pod is automatically assigned a default service account in its namespace. Key functions include:

• Providing an identity for the pod within the cluster.
• Being associated with RBAC roles to grant API permissions.
• Automatically mounting a Secret containing a API token into the pod at /var/run/secrets/kubernetes.io/serviceaccount.

Service Accounts and their tokens are managed by Kubernetes and are a primary method for in-cluster authentication.

The Kubernetes authorization system that regulates access to resources, including Secrets, based on user roles. Critical components for Secret security are:

• Roles and ClusterRoles: Define permissions (e.g., get, list, create Secrets).
• RoleBindings and ClusterRoleBindings: Bind a role to subjects (users, groups, or ServiceAccounts).

Proper RBAC configuration is mandatory to enforce the principle of least privilege, ensuring pods and users can only access the Secrets they explicitly require.

A Kubernetes feature that encrypts Secret data stored in the cluster's backing datastore (etcd). Without this, anyone with access to the etcd data can read Secret contents.

• Configured via an EncryptionConfiguration file provided to the Kubernetes API server.
• Supports multiple encryption providers (e.g., AES-CBC, AES-GCM, or integration with external Key Management Services like KMS).

This is a critical defense-in-depth layer, protecting Secret data from administrators with raw etcd access and in case of disk theft.