PodSecurityPolicy migration
For Kubernetes ≥ v1.25. PodSecurityPolicy (PSP) is removed. Now you can use Kubewarden for admission control on your Kubernetes clusters.
Kubewarden has separate policies to achieve the same goal as a monolithic PSP configuration. Each Kubewarden policy definition functions as a different configuration section in the specification of a PSP. The mapping of PSP configuration fields to their respective Kubewarden policies is in the mapping table below.
With Kubewarden, operators have granular control of policy configuration in their clusters.
With a Kubewarden instance, you can deploy policies to replace the
PodSecurityPolicy
object. We consider these rules in this example:
-
a PSP disabling privileged escalation
-
privileged containers
-
blocking pods running as root
-
forcing a particular user group
-
blocking host namespaces
-
allowing a pod to use only port 443
The YAML definition of this PSP is:
PSP YAML definition
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
allowPrivilegeEscalation: false
runAsUser:
rule: MustRunAsNonRoot
supplementalGroups:
rule: MustRunAs
ranges:
- min: 1000
max: 65535
privileged: false
hostNetwork: false
hostIPC: false
hostPID: false
hostPorts:
- min: 443
max: 443
Kubewarden replacements for PSP
Now we will create Kubewarden policies to achieve the same goal. You enforce each rule with a separate Kubewarden policy. So, in this example, you need a separate policy for the enforcement of each of:
-
privileged escalation
-
user and group configuration
-
host namespaces
-
privileged container configuration.
Blocking container privilege escalation
You can deploy a policy as shown below:
kubectl command for policy deployment
$ kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1
kind: ClusterAdmissionPolicy
metadata:
name: psp-allow-privilege-escalation
spec:
module: ghcr.io/kubewarden/policies/allow-privilege-escalation-psp:v0.2.6
rules:
- apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
operations:
- CREATE
- UPDATE
mutating: false
settings:
default_allow_privilege_escalation: false
EOF
In that command, we have specified default_allow_privilege_escalation
to be
false
. This policy restricts pods that try to run with more privileges than
the parent container.
Output from kubectl that attempts to raise privilege
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
securityContext:
allowPrivilegeEscalation: true
- name: sidecar
image: sidecar
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-allow-privilege-escalation.kubewarden.admission" denied the request: one of the containers has privilege escalation enabled
User and group configuration
Now, to enforce the user and group configuration, you can use the
" class="bare">https://github.com/kubewarden/user-group-psp-policy
kubectl command to use user-group-psp-policy
shell $ kubectl apply -f - <<EOF apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-user-group spec: module: ghcr.io/kubewarden/policies/user-group-psp:v0.4.9 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: true settings: run_as_user: rule: MustRunAsNonRoot supplemental_groups: rule: MustRunAs ranges: - min: 1000 max: 65535 EOF
You should configure the policy with mutation: true
. It’s required because
the policy will add
supplementalGroups
when the user does not define them.
So, now users cannot deploy pods running as root:
Example output where runAsNonRoot: false
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
securityContext:
runAsNonRoot: false
runAsUser: 0
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-user-group-fb836.kubewarden.admission" denied the request: RunAsNonRoot should be set to true
Example output where runAsUser: 0
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
securityContext:
runAsNonRoot: true
runAsUser: 0
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-user-group-fb836.kubewarden.admission" denied the request: Invalid user ID: cannot run container with root ID (0)
This example below shows the addition of a supplemental group, despite it not being defined by us.
Example addition of a supplemental group
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
EOF
pod/nginx created
$ kubectl get pods -o json nginx | jq ".spec.securityContext"
{
"supplementalGroups": [
10000
]
}
Privileged container configuration
You need to replace the older PSP configuration that blocks privileged containers. It’s necessary to deploy the pod-privileged policy. This policy does not need any settings. Once running, it will block privileged pods.
Applying the pod-privileged-policy
$ kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1
kind: ClusterAdmissionPolicy
metadata:
name: psp-privileged
spec:
module: ghcr.io/kubewarden/policies/pod-privileged:v0.2.7
rules:
- apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
operations:
- CREATE
- UPDATE
mutating: false
settings: null
EOF
To test the policy, we can try running a pod with privileged configuration enabled:
Pod run with privileged configuration enabled
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
- name: sleeping-sidecar
image: alpine
command: ["sleep", "1h"]
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-privileged.kubewarden.admission" denied the request: Privileged container is not allowed
Host namespace configuration
To finish the PSP migration exercise, you need to disable host namespace
sharing. For that, we shall be using the host-namespace-psp
policy. It allows
the cluster administrator to block IPC, PID, and network namespaces
individually. It also sets the ports that the pods can be open on, on the host
IP.
Disabling namespace sharing and setting ports
$ kubectl apply -f - <<EOF
apiVersion: policies.kubewarden.io/v1
kind: ClusterAdmissionPolicy
metadata:
name: psp-hostnamespaces
spec:
module: ghcr.io/kubewarden/policies/host-namespaces-psp:v0.1.6
rules:
- apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
operations:
- CREATE
- UPDATE
mutating: false
settings:
allow_host_ipc: false
allow_host_pid: false
allow_host_ports:
- min: 443
max: 443
allow_host_network: false
EOF
We can validate the policy. The pod should not be able to share host namespaces:
Blocking namespace example
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
hostIPC: true
hostNetwork: false
hostPID: false
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
- name: sleeping-sidecar
image: alpine
command: ["sleep", "1h"]
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-hostnamespaces.kubewarden.admission" denied the request: Pod has IPC enabled, but this is not allowed
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
hostIPC: false
hostNetwork: true
hostPID: false
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
- name: sleeping-sidecar
image: alpine
command: ["sleep", "1h"]
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-hostnamespaces.kubewarden.admission" denied the request: Pod has host network enabled, but this is not allowed
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
hostIPC: false
hostNetwork: false
hostPID: true
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
- name: sleeping-sidecar
image: alpine
command: ["sleep", "1h"]
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-hostnamespaces.kubewarden.admission" denied the request: Pod has host PID enabled, but this is not allowed
In this last example, the pod should only be able to expose port 443.
If other ports are configured in hostPorts
then an error should happen.
Attempting to use port 80 in hostPorts
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
hostPort: 80
- name: sleeping-sidecar
image: alpine
command: ["sleep", "1h"]
EOF
Error from server: error when creating "STDIN": admission webhook "clusterwide-psp-hostnamespaces.kubewarden.admission" denied the request: Pod is using unallowed host ports in containers
Mapping Kuberwarden policies to PSP fields
This table maps PSP configuration fields to corresponding Kubewarden policies.
PSP field | Kubewarden equivalent policy |
---|---|
PSP migration script
The Kubewarden team has developed a script for PSP migration. It uses the migration tool from AppVia. The AppVia tool reads a PSP YAML configuration. It then generates the corresponding policies. It does this for Kubewarden and other policy engines.
The AppVia migration tool is out of control of the Kuberwarden maintainers. This means that it’s possible it generates out-of-date Kubewarden policies. Use with caution. We need a pull request for AppVia for which work is ongoing. Contact us for more information if you need to. |
The script is available in the Kubewarden
utils
repository. It downloads the AppVia migration tool into the working directory
to use. It processes the PSPs defined in the kubectl
default context. Then it
prints the Kuberwarden policies definitions on the standard output. Users can
redirect the content to a file or to kubectl
directly.
This script only works in Linux x86_64 machines. |
Let’s take a look at an example. In a cluster with the PSP:
-
blocking access to host namespaces
-
blocking privileged containers
-
not allowing privilege escalation
-
dropping container capabilities
-
listing the allowed volume types
-
defining the allowed users and groups to be used
-
controlling the supplemental group applied to volumes
-
forcing containers to run in a read-only root filesystem
The following YAML could be used.
The PSP configuration
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
hostNetwork: false
hostIPC: false
hostPID: false
hostPorts:
- min: 80
max: 8080
privileged: false
# Required to prevent escalations to root.
allowPrivilegeEscalation: false
requiredDropCapabilities:
- ALL
# Allow core volume types.
volumes:
- "configMap"
- "emptyDir"
- "projected"
- "secret"
- "downwardAPI"
# Assume that ephemeral CSI drivers & persistentVolumes set up by the cluster admin are safe to use.
- "csi"
- "persistentVolumeClaim"
- "ephemeral"
runAsUser:
# Require the container to run without root privileges.
rule: "MustRunAsNonRoot"
seLinux:
# This policy assumes the nodes are using AppArmor rather than SELinux.
rule: "RunAsAny"
supplementalGroups:
rule: "MustRunAs"
ranges:
# Forbid adding the root group.
- min: 1
max: 65535
fsGroup:
rule: "MustRunAs"
ranges:
# Forbid adding the root group.
- min: 1
max: 65535
readOnlyRootFilesystem: true
Kubewarden policies can be applied directly to a cluster using the following command:
$ ./psp-to-kubewarden | kubectl apply -f -
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
clusteradmissionpolicy.policies.kubewarden.io/psp-privileged-82bf2 created
clusteradmissionpolicy.policies.kubewarden.io/psp-readonlyrootfilesystem-b4a55 created
clusteradmissionpolicy.policies.kubewarden.io/psp-hostnamespaces-a25a2 created
clusteradmissionpolicy.policies.kubewarden.io/psp-volumes-cee05 created
clusteradmissionpolicy.policies.kubewarden.io/psp-capabilities-34d8e created
clusteradmissionpolicy.policies.kubewarden.io/psp-usergroup-878b0 created
clusteradmissionpolicy.policies.kubewarden.io/psp-fsgroup-3b08e created
clusteradmissionpolicy.policies.kubewarden.io/psp-defaultallowprivilegeescalation-b7e87 created
If users want to inspect the policies before applying, it’s possible to redirect the content to a file or review it directly in the console
To store the generated policies and view them:`
$ ./psp-to-kubewarden > policies.yaml && cat policies.yaml
`shell $ ./psp-to-kubewarden > policies.yaml $ cat policies.yaml --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-privileged-eebb9 spec: module: registry://ghcr.io/kubewarden/policies/pod-privileged:v0.2.7 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: null --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-readonlyrootfilesystem-34d7c spec: module: registry://ghcr.io/kubewarden/policies/readonly-root-filesystem-psp:v0.1.6 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: null --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-hostnamespaces-41314 spec: module: registry://ghcr.io/kubewarden/policies/host-namespaces-psp:v0.1.6 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: allow_host_ipc: false allow_host_pid: false allow_host_ports: - max: 8080 min: 80 allow_host_network: false --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-volumes-2fd34 spec: module: registry://ghcr.io/kubewarden/policies/volumes-psp:v0.1.11 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: allowedTypes: - configMap - emptyDir - projected - secret - downwardAPI - csi - persistentVolumeClaim - ephemeral --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-capabilities-340fe spec: module: registry://ghcr.io/kubewarden/policies/capabilities-psp:v0.1.13 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: allowed_capabilities: [] required_drop_capabilities: - ALL --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-usergroup-19f7a spec: module: registry://ghcr.io/kubewarden/policies/user-group-psp:v0.4.9 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: run_as_user: rule: MustRunAsNonRoot supplemental_groups: ranges: - max: 65535 min: 1 rule: MustRunAs --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-fsgroup-52337 spec: module: registry://ghcr.io/kubewarden/policies/allowed-fsgroups-psp:v0.1.10 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: ranges: - max: 65535 min: 1 rule: MustRunAs --- apiVersion: policies.kubewarden.io/v1 kind: ClusterAdmissionPolicy metadata: name: psp-defaultallowprivilegeescalation-6f11b spec: module: registry://ghcr.io/kubewarden/policies/allow-privilege-escalation-psp:v0.2.6 rules: - apiGroups: - "" apiVersions: - v1 resources: - pods operations: - CREATE - UPDATE mutating: false settings: default_allow_privilege_escalation: false
The policy names are generated by the PSP migration tool. You may want to change the name to something more meaningful. |