SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) architecture

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) is a Kubernetes policy engine. It uses policies written in a programming language of your choosing. This language must generate a WebAssembly binary for SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) to use.

What is a policy?

A policy is an Open Container Initiative (OCI) artifact containing a WebAssembly module, the policy code, and the metadata required by PolicyServer performing admission request validations and mutations.

In the same manner as Kubernetes, SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) uses the terms 'PolicyServer' when discussing the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policy server and policy-server when discussing Pod or Deployment of a SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) PolicyServer.

Design principles

Making use of core Kubernetes features

The team designed SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) to use core features of Kubernetes, without reinventing the wheel. The project utilizes a combination of:

  • Kubernetes Controllers

  • Custom Resource Definitions (CRDs)

  • Webhooks (Validation and Mutating)

  • the Control Plane’s event notification system

Effectively uses Kubernetes architecture

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) operates seamlessly within the Kubernetes ecosystem. At its core, the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) controller is a Kubernetes controller, monitoring SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) Custom Resource Definitions (CRDs) and configuring Kubernetes resources to execute them. This integration ensures that SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) uses the built-in Kubernetes mechanisms, such as controllers and CRDs, to watch, manage, and apply security policies efficiently.

Extensible policy definition

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) employs CRDs to define and manage SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) resources, which specify the rules for admission request validations. This design enables users to extend Kubernetes' capabilities with custom admission controls, ensuring that security and compliance policy enforcement is consistent across the cluster.

Direct admission control

When setup by the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) controller, the policy-server Service receives admission requests directly from the Kubernetes control plane, using ValidationWebhooks and MutatingWebhooks. This direct interaction streamlines the admission control process, reducing latency and increasing efficiency in policy enforcement.

WebAssembly offers a sand-boxed execution environment, ensuring policies run in isolation, thus enhancing the security and stability of the policy enforcement mechanism. This isolation prevents policies from interfering with each other or with the host system, mitigating the risk of malicious code execution. WebAssembly is portable and efficient, enabling policies to run across different environments without modification. This cross-platform compatibility ensures that SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies are versatile, and can be distributed and executed in diverse Kubernetes clusters

OCI based policy artifacts

Policies in SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) are OCI (Open Container Initiative) artifacts. This standardization makes the distribution and versioning of policies easier, Policies contain both the WebAssembly modules for enforcement logic, and metadata necessary for the PolicyServer’s operation. Leveraging OCI artifacts promotes interoperability and ease of management within cloud ecosystems.

Fine-grained policy application

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) associates policies with their own 'validation' or 'mutating' webhook, allowing for fine-grained application of admission controls. This flexibility enables administrators to tailor the enforcement of policies according to specific needs, enhancing the security and compliance posture of the Kubernetes cluster.

The SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) stack

The SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) consists of these components:

  • SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) Custom Resources are Kubernetes Custom Resources that simplify the process of managing policies.

    SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) integrates with Kubernetes using Dynamic Admission Control. In particular, SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) operates as a Kubernetes Admission Webhook. The policy-server is the Webhook endpoint called by the Kubernetes API server to validate requests.

  • The SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) controller is a Kubernetes controller that reconciles SUSE® Rancher Prime: Admission Policy Manager (Kubewarden)'s Custom Resources. This controller creates parts of the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) stack. It also translates SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) configuration into Kubernetes directives.

    The kubewarden-controller registers the needed MutatingWebhookConfiguration or ValidatingWebhookConfiguration objects with the Kubernetes API server.

  • SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies are WebAssembly modules holding the validation or mutation logic. WebAssembly modules have detailed documentation in the writing policies sections.

  • The PolicyServer receives requests for validation. It validates the requests by executing SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies.

  • The audit scanner inspects the resources already in the cluster. It identifies those violating SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies.

    Audit scanner constantly checks the resources declared in the cluster, flagging the ones that no longer adhere with the deployed SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies.

    %%{ init: { "flowchart": { "htmlLabels": false, } } }%% graph LR subgraph " " direction LR subgraph " " direction LR k8s(("Kubernetes")) registry[("OCI registry")] end subgraph kw["`**Kubewarden**`"] controller("`**KW controller**`") subgraph policy-server["`**policy-server**`"] direction LR kw-policy-1{{"Policy 1"}} kw-policy-2{{"Policy 2"}} kw-policy-3{{"Policy 3"}} end webhooks(["ValidationWebhooks and\nMutatingWebhooks"]) audit-scanner["KW audit scanner"] end end policy-server -->|"downloads\npolicies from"| registry controller -->|"watches for\nevents"| k8s controller -->|"creates"| webhooks controller -->|"creates\npolicy-server\ninstances"| policy-server k8s -. "sends admission\nrequests using" .-> webhooks webhooks -. "sent admission\nrequests from K8s" .-> policy-server audit-scanner -->|"sends audit\nadmission requests"| policy-server
    Figure 1. {product-name} architecture

The journey of a SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policy

Default PolicyServer

On a new cluster, the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) components defined are:

  • Custom Resource Definitions (CRD)

  • The kubewarden-controller Deployment

  • A PolicyServer Custom Resource named default.

When the kubewarden-controller notices the default PolicyServer resource, it creates a policy-server deployment of the PolicyServer component.

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) works as a Kubernetes Admission Webhook. Kubernetes specifies using Transport Layer Security (TLS) to secure all Webhook endpoints. The kubewarden-controller sets up this secure communication by:

  1. Generating a self-signed Certificate Authority

  2. Use this CA to generate a TLS certificate key for the policy-server Service.

These objects are all stored as Secret resources in Kubernetes.

Finally, kubewarden-controller creates the policy-server Deployment and a Kubernetes ClusterIP Service to expose it inside the cluster network.

Defining the first policy

A policy must define which policy-server it must run on. It binds to a policy-server instance. You can have different policies with the same Wasm module and settings running in many PolicyServers. However, you can’t have a single policy definition that runs in many PolicyServers.

The kubewarden-controller notices the new ClusterAdmissionPolicy resource and so finds the bound policy-server and reconciles it.

Reconciliation of a policy-server

When creating, modifying or deleting a ClusterAdmissionPolicy or AdmissionPolicy, a reconciliation loop activates in kubewarden-controller, for the policy-server owning the policy. This reconciliation loop creates a ConfigMap with all the policies bound to the policy-server. Then the Deployment rollout of the policy-server starts. It results in starting the new policy-server instance with the updated configuration.

At start time, the policy-server reads its configuration from the ConfigMap and downloads all the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies specified. You can download SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies from remote HTTP servers and container registries.

You use policy settings parameters to tune a policies' behavior. After startup and policy download the policy-server checks the policy settings provided by the user are valid.

The policy-server validates policy settings by invoking the validate_setting function exposed by each policy. There is further documentation in the specification reference section of the documentation.

If one or more policies received wrong configuration parameters, from the policy specification provided by the user, then any admission requests evaluated by that policy return an error.

When SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) has configured all policies, the policy-server spawns a pool of worker threads to evaluate incoming requests using the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies specified by the user.

Finally, the policy-server starts a HTTPS server, listening to incoming validation requests. SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) uses the TLS key and certificate created by the SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) controller to secure the web server.

The web server exposes each policy by a dedicated path following the naming convention: /validate/<policy ID>.

Making Kubernetes aware of the policy

All policy-server instances have a Readiness Probe, that kubewarden-controller uses to check when the policy-server Deployment is ready to evaluate an AdmissionReview.

Once SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) marks the policy-server deployment as 'uniquely reachable' or Ready, the kubewarden-controller makes the Kubernetes API server aware of the new policy. This is by creating either a MutatingWebhookConfiguration or a ValidatingWebhookConfiguration object. In this context, 'uniquely reachable', means that all the PolicyServer instances in the cluster have the latest policy configuration installed. The distinction, is a fine point, but is necessary, due to how roll-out of PolicyServers works. It’s possible to have the same policy, on different PolicyServers with different configurations.

Each policy has a dedicated MutatingWebhookConfiguration or ValidatingWebhookConfiguration pointing to the Webhook endpoint served by policy-server. The endpoint is reachable at the /validate/<policy ID> URL.

Policy in action

Now that all the necessary plumbing is complete, Kubernetes starts sending Admission Review requests to the right policy-server endpoints.

A policy-server receives the Admission Request object and, based on the endpoint that received the request, uses the correct policy to evaluate it.

SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) evaluates each policy inside its own dedicated WebAssembly sand-box. The communication between a policy-server instance (the "host") and the WebAssembly policy (the "guest") uses the waPC communication protocol. The protocol description is part of the writing policies documentation. Policies can also use the interfaces provided by the Web Assembly System Interface (WASI).

How SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) handles many PolicyServer and policies

A cluster can have many PolicyServers and SUSE® Rancher Prime: Admission Policy Manager (Kubewarden) policies defined. There are benefits of having many PolicyServers:

  • You can isolate noisy namespaces or tenants, those generating many policy evaluations, from the rest of the cluster so as not to adversely affect other cluster operations.

  • You can run mission-critical policies in a dedicated PolicyServer pool, making your infrastructure more resilient.

A PolicyServer resource defines each policy-server and a ClusterAdmissionPolicy or AdmissionPolicy resource defines each policy.

A ClusterAdmissionPolicy and an AdmissionPolicy bind to a policy-server. Any ClusterAdmissionPolicy not specifying a policy-server binds to the default PolicyServer. If a ClusterAdmissionPolicy references a policy-server that doesn’t exist, its state is unschedulable.

Each policy-server defines many validation endpoints, one for each policy defined in its configuration file. You can load the same policy many times, with different configuration parameters.

The ValidatingWebhookConfiguration and MutatingWebhookConfiguration resources make the Kubernetes API server aware of these policies. Then kubewarden-controller keeps the API server and configuration resources in synchronization.

The Kubernetes API server dispatches incoming admission requests to the correct validation endpoint exposed by policy-server.