Replacing Internal Certificates

Internal Communication and Certificates

SUSE® Security includes default self-signed certificates for encryption for the Manager (console/UI access), Controller (REST API, internal), Enforcer (internal), and Scanner (internal) communications.

These certificates can be replaced by your own to further harden communication. For replacing certificates used by external access to SUSE® Security (i.e, browser to the Manager, or REST API to the Controller), please see this section. See below for replacing the certificates used in internal communication between SUSE® Security containers.

Replacing certificates is recommended to be performed only during initial deployment of SUSE® Security. Replacing on a running cluster (even with rolling upgrade) may result in an unstable state where SUSE® Security pods are unable to communicate with each other due to a mismatch in certificates, and DATA LOSS may occur.

Replacing Certificates Used in Internal Communications of SUSE® Security

Replace the internal encryption files ca.crt, tls.key, tls.crt as follows:

  • Create a new ca.cfg file with your favorite editor:

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = California
L = San Jose
O = {product-name} Inc.
OU = Neuvector
CN = Neuvector
[v3_req]
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = Neuvector

For additional information on ca.cfg, see https://open-docs.neuvector.com/configuration/console/replacecert.

  • Choose your scenario from the three options below:

  • New certificate

  • Update current certificate with SANs

  • Regenarate certificate files and add SANs

If your certificate is about to expire and you need to generate a new one, follow the steps below: * Delete the old ca.crt, tls.key, tls.crt, kubernetes secret, and generate new ones:

+

kubectl delete secret internal-cert -n neuvector
openssl genrsa -out ca.key 2048
openssl req -x509 -sha256 -new -nodes -key ca.key -days 3650 -out ca.crt
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca.cfg
openssl x509 -in tls.crt -text
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key --from-file=tls.crt --from-file=ca.crt

  • Then edit the Controller, Enforcer, and Scanner deployment yamls, adding:

          containers:
        - name: neuvector-controller/enforcer/scanner-pod
          volumeMounts:
            - mountPath: /etc/neuvector/certs/internal/cert.key
              name: internal-cert
              readOnly: true
              subPath: tls.key
            - mountPath: /etc/neuvector/certs/internal/cert.pem
              name: internal-cert
              readOnly: true
              subPath: tls.crt
            - mountPath: /etc/neuvector/certs/internal/ca.cert
              name: internal-cert
              readOnly: true
              subPath: ca.crt
      volumes:
        - name: internal-cert
          secret:
            defaultMode: 420
            secretName: internal-cert

Then proceed to deploy SUSE® Security as before. You can also shell into the controller/enforcer/scanner pods to confirm that the ca.crt, tls.key, tls.crt files are the customized ones and that the SUSE® Security communications are working using the new certificates.

Example patch commands for controller (change namespace to cattle-neuvector-system if needed, and modify for use on enforcer, scanner)
NAMESPACE=neuvector

kubectl patch deployment -n $\{NAMESPACE} neuvector-controller-pod --type='json' -p='[{"op": "add", "path": "/spec/template/spec/volumes/-", "value": {"name": "internal-cert", "secret": {"defaultMode": 420, "secretName": "internal-cert"}} } ]'

kubectl patch deployment -n $\{NAMESPACE} neuvector-controller-pod --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/volumeMounts", "value": [{"mountPath": "/etc/neuvector/certs/internal/cert.key", "name": "internal-cert", "readOnly": true, "subPath": "cert.key"}, {"mountPath": "/etc/neuvector/certs/internal/cert.pem", "name": "internal-cert", "readOnly": true, "subPath": "cert.pem"}, {"mountPath": "/etc/neuvector/certs/internal/ca.cert", "name": "internal-cert", "readOnly": true, "subPath": "ca.cert"} ] } ]'

If your certificate files were created before SUSE® Security version 5.3, you need to upate the certificate with at least one Subject Alternative Name, or SAN for short. If you still have the files ca.key and ca.crt accessible, run the commands as follows:

kubectl delete secret internal-cert -n neuvector
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca-new.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca-new.cfg
openssl x509 -in tls.crt -text
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key --from-file=tls.crt --from-file=ca.crt

Once the certificate files have been updated, restart the deployments in order to use the updated certificate:

kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod

If your certificate files were created before SUSE® Security version 5.3, you need to upate the certificate with at least one Subject Alternative Name, or SAN for short. If you don´t have the files ca.key and ca.crt anymore, follow the steps below:

  • Backup your original certificate

    kubectl get secret internal-cert -o yaml > internal-cert.yaml

  • Export the existing internal-cert

    kubectl get secret internal-cert -o json | jq -r '.data."ca.crt"' | base64 -d > old-ca.crt
kubectl get secret internal-cert -o json | jq -r '.data."tls.crt"' | base64 -d > old-tls.crt
kubectl get secret internal-cert -o json | jq -r '.data."tls.key"' | base64 -d > old-tls.key

  • Create new certificate files and internal certificates

    openssl genrsa -out ca.key 2048
openssl req -x509 -sha256 -new -nodes -key ca.key -days 3650 -out ca.crt
openssl genrsa -out tls.key 2048
openssl req -new -key tls.key -sha256 -out cert.csr -config ca.cfg
openssl req -in cert.csr -noout -text
openssl x509 -req -sha256 -in cert.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -days 3650 -extensions 'v3_req' -extfile ca.cfg
openssl x509 -in tls.crt -text

  • Merge the old and new ca.crt files

    cat old-ca.crt > /tmp/ca.crt cat ca.crt >> /tmp/ca.crt

  • Update the Kubernetes secret with the merged ca.crt

    kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=old-tls.key --from-file=tls.crt=old-tls.crt --from-file=ca.crt=/tmp/ca.crt

  • Restart the deployments in order to use the updated certificate

    kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod

  • Wait for the restart to complete

    kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod

  • Make sure the console can be accessed and controllers are all online.* Update the Kubernetes secret with the new tls.key

    kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=tls.key --from-file=tls.crt=tls.crt --from-file=ca.crt=/tmp/ca.crt

  • Restart the deployments in order to use the updated certificate

    kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod

  • Wait for the restart to complete

    kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod

  • Make sure the console can be accessed and controllers are all online.* Update the Kubernetes secret with the new ca.crt

    kubectl delete secret internal-cert -n neuvector
kubectl create secret generic internal-cert -n neuvector --from-file=tls.key=tls.key --from-file=tls.crt=tls.crt --from-file=ca.crt=ca.crt

  • Restart the deployments in order to use the updated certificate

    kubectl rollout restart deployment neuvector-controller-pod
kubectl rollout restart deployment neuvector-scanner-pod
kubectl rollout restart deployment neuvector-registry-adapter-pod
kubectl rollout restart ds neuvector-enforcer-pod

  • Wait for the restart to complete

    kubectl rollout status deployment neuvector-controller-pod
kubectl rollout status deployment neuvector-scanner-pod
kubectl rollout status deployment neuvector-registry-adapter-pod
kubectl rollout status ds neuvector-enforcer-pod

  • Make sure the console can be accessed and controllers are all online.

Updating/Deploying with Helm

As of Helm chart 2.4.1 we can now manage the internal certificate install. The chart values.yaml should be reviewed for all the settings. The below example uses RKE2, standard Ingress and installer certificates.

# add chart
helm repo add neuvector https://neuvector.github.io/neuvector-helm/

# update chart
helm repo update

# add domain for ingress
export domain=awesome.sauce

# run the helm
helm upgrade -i neuvector -n neuvector neuvector/core --create-namespace  --set imagePullSecrets=regsecret --set k3s.enabled=true --set k3s.runtimePath=/run/k3s/containerd/containerd.sock --set manager.ingress.enabled=true --set manager.ingress.host=neuvector.$domain --set manager.svc.type=ClusterIP --set controller.pvc.enabled=true --set controller.pvc.capacity=500Mi --set controller.internal.certificate.secret=internal-cert --set cve.scanner.internal.certificate.secret=internal-cert --set enforcer.internal.certificate.secret=internal-cert