- How it works
- Secrets Store CSI Driver
- Provider for the Secrets Store CSI Driver
- Custom Resource Definitions (CRDs)
The diagram below illustrates how Secrets Store CSI volume works:
Similar to Kubernetes secrets, on pod start and restart, the Secrets Store CSI driver communicates with the provider using gRPC to retrieve the secret content from the external Secrets Store specified in the
SecretProviderClass custom resource. Then the volume is mounted in the pod as
tmpfs and the secret contents are written to the volume.
On pod delete, the corresponding volume is cleaned up and deleted.
The Secrets Store CSI Driver is a daemonset that facilitates communication with every instance of Kubelet. Each driver pod has the following containers:
node-driver-registrar: Responsible for registering the CSI driver with Kubelet so that it knows which unix domain socket to issue the CSI calls on. This sidecar container is provider by the Kubernetes CSI team. See doc for more details.
secrets-store: Implements the CSI
Nodeservice gRPC services described in the CSI specification. It’s responsible for mount/unmount the volumes during pod creation/deletion. This component is developed and maintained in this repo.
liveness-probe: Responsible for monitoring the health of the CSI driver and reports to Kubernetes. This enables Kubernetes to automatically detect issues with the driver and restart the pod to try and fix the issue. This sidecar container is provider by the Kubernetes CSI team. See doc for more details.
The CSI driver communicates with the provider using gRPC to fetch the mount contents from external Secrets Store. Refer to doc for more details on the how to implement a provider for the driver and criteria for supported providers.
Currently supported providers:
The Secrets Store CSI Driver daemonset runs as
root in a
privileged pod. This is because the daemonset is
responsible for creating new
tmpfs filesystems and
mounting them into existing pod filesystems within the node’s
root is necessary for the
mount syscall and other filesystem operations and
privileged is required for
mountPropagation: Bidirectional to modify other running pod’s filesystems.
The provider plugins are also required to run as
privileged should not be necessary). This is because
the provider plugin must create a unix domain socket in a
hostPath for the driver to connect to.
Further, service account tokens for pods that require secrets may be forwarded from the kubelet process to the driver and then to provider plugins. This allows the provider to impersonate the pod when contacting the external secret API.
Note: On Windows hosts secrets will be written to the node’s filesystem which may be persistent storage. This
contrasts with Linux where a
tmpfs is used to try to ensure that secret material is never persisted.
Note: Kubernetes 1.22 introduced a way to configure nodes to
use swap memory, however if this is used then secret
material may be persisted to the node’s disk. To ensure that secrets are not written to persistent disk ensure
failSwapOn is set to
true (which is the default).
Anyone who has access to the namespace or its resources can exec and view the secrets. However, this behavior is expected, as entities with such access are expected to perform these operations. One potential solution is to disable exec if a more stringent security posture is required. Alternatively, using distroless images for applications can mitigate this, as exec won’t work. A similar argument can be made for individuals with access to the underlying infrastructure or nodes, who can access the secrets by SSHing into the node. However, typical end users do not have this level of access. If an end user does gain such access, it indicates a compromised infrastructure/cluster, and the recommended solution is to restrict access to the cluster/infrastructure.
Encrypting mounted content can be the solution to further protect the secrets. However, this introduces additional operational overhead, such as managing encryption keys and addressing key rotation. Key management becomes a crucial aspect similar to secrets management.
When we look from the perspective of an application and what access does it have, for instance, the Ingress Controller application which requires cluster-wide access to Kubernetes Secrets. If a component like Ingress is compromised, it could jeopardize all secrets in the cluster. This is where the Secrets Store CSI driver proves valuable, as it can mount/sync only the necessary TLS certificates on the Ingress Pod, reducing the blast radius.
SecretProviderClass is a namespaced resource in Secrets Store CSI Driver that is used to provide driver configurations and provider-specific parameters to the CSI driver.
SecretProviderClass custom resource should have the following components:
provider: vault # accepted provider options: akeyless or azure or vault or gcp
parameters: # provider-specific parameters
Refer to the provider docs for required provider specific parameters.
Here is an example of a
SecretProviderClass in the pod volumes when using the CSI driver:
- name: secrets-store-inline
SecretProviderClassneeds to be created in the same namespace as the pod.
SecretProviderClassPodStatus is a namespaced resource in Secrets Store CSI Driver that is created by the CSI driver to track the binding between a pod and
SecretProviderClassPodStatus contains details about the current object versions that have been loaded in the pod mount.
SecretProviderClassPodStatus is created by the CSI driver in the same namespace as the pod and
SecretProviderClass with the name
<pod name>-<namespace>-<secretproviderclass name>.
Here is an example of a
- apiVersion: v1
- id: secret/secret1
- id: secret/secret2
The pod for which the
SecretProviderClassPodStatus was created is set as owner. When the pod is deleted, the
SecretProviderClassPodStatus resources associated with the pod get automatically deleted.