SUSE Cloud Application Platform is a software platform for cloud-native applications based on Cloud Foundry Application Runtime (cf-operator, KubeCF, and Stratos) with additional supporting components.

SUSE Cloud Application Platform describes the complete software stack, including the operating system, Kubernetes, and KubeCF.

SUSE Cloud Application Platform is comprised of cf-operator (cf-operator), KubeCF (kubecf), the Stratos Web user interface, and Stratos Metrics.

The Cloud Foundry code base provides the basic functionality. KubeCF differentiates itself from other Cloud Foundry distributions by running in Linux containers managed by Kubernetes, rather than virtual machines managed with BOSH, for greater fault tolerance and lower memory use.

All Docker images for the SUSE Linux Enterprise builds are hosted on registry.suse.com. These are the commercially-supported images. (Community-supported images for openSUSE are hosted on Docker Hub.) Product manuals on https://documentation.suse.com/suse-cap/2/ refer to the commercially-supported SUSE Linux Enterprise version.

Cloud Application Platform is designed to run on any Kubernetes cluster as described in Section 5.2, “Platform Support”. This guide describes how to deploy it:

SUSE Cloud Application Platform serves different but complementary purposes for operators and application developers.

For operators, the platform is:

For developers, the platform:

The principle interface and API for deploying applications to SUSE Cloud Application Platform is KubeCF. Most Cloud Foundry distributions run on virtual machines managed by BOSH. KubeCF runs in SUSE Linux Enterprise containers managed by Kubernetes. Containerizing the components of the platform itself has these advantages:

SUSE Cloud Application Platform uses cf-operator, a Kubernetes Operator deployed via a Helm chart, to install custom resource definitions that convert BOSH releases into Kubernetes resources, such as Pod, Deployment, and StatefulSet. This is made possible by leveraging KubeCF, a version of Cloud Foundry deployed as Helm chart.

The following figures illustrate the main structural concepts of SUSE Cloud Application Platform. Figure 1.1, “Cloud Platform Comparisons” shows a comparison of the basic cloud platforms:

SUSE CaaS Platform is a Container as a Service platform, and SUSE Cloud Application Platform is a PaaS.

Figure 1.1: Cloud Platform Comparisons #

Figure 1.2, “Containerized Platforms” illustrates how SUSE Cloud Application Platform containerize the platform itself on top of a cloud provider.

Figure 1.2: Containerized Platforms #

Figure 1.3, “SUSE Cloud Application Platform Stack” shows the relationships of the major components of the software stack. SUSE Cloud Application Platform runs on Kubernetes, which in turn runs on multiple platforms, from bare metal to various cloud stacks. Your applications run on SUSE Cloud Application Platform and provide services.

Figure 1.3: SUSE Cloud Application Platform Stack #

SUSE Cloud Application Platform is designed to run on any Kubernetes system that meets the following requirements:

Schedulers such as Diego and Eirini, and stacks such as cflinuxfs3 or sle15, have different memory requirements for applications. Not every combination is tested so there is no universal memory setting for Cloud Application Platform, and because it depends on the application deployed, it is up to the user to adjust the setting based on their application.

During deployment, pods are spawned over time, starting with a single pod whose name stars with ig-. This pod will eventually disappear and will be replaced by other pods whose progress then can be followed as usual.

The whole process can take around 20—30 minutes to finish.

The initial stage may look like this:

tux > kubectl get pods --namespace kubecf
ig-kubecf-f9085246244fbe70-jvg4z   1/21    Running             0          8m28s

Later the progress may look like this:

NAME                        READY   STATUS       RESTARTS   AGE
adapter-0                   4/4     Running      0          6m45s
api-0                       0/15    Init:30/63   0          6m38s
bits-0                      0/6     Init:8/15    0          6m34s
bosh-dns-7787b4bb88-2wg9s   1/1     Running      0          7m7s
bosh-dns-7787b4bb88-t42mh   1/1     Running      0          7m7s
cc-worker-0                 0/4     Init:5/9     0          6m36s
credhub-0                   0/5     Init:6/11    0          6m33s
database-0                  2/2     Running      0          6m36s
diego-api-0                 6/6     Running      2          6m38s
doppler-0                   0/9     Init:7/16    0          6m40s
eirini-0                    9/9     Running      0          6m37s
log-api-0                   0/7     Init:6/13    0          6m35s
nats-0                      4/4     Running      0          6m39s
router-0                    0/5     Init:5/11    0          6m33s
routing-api-0               0/4     Init:5/10    0          6m42s
scheduler-0                 0/8     Init:8/17    0          6m35s
singleton-blobstore-0       0/6     Init:6/11    0          6m46s
tcp-router-0                0/5     Init:5/11    0          6m37s
uaa-0                       0/6     Init:8/13    0          6m36s

Release names (for example, when you run helm install RELEASE_NAME) have a maximum length of 36 characters.

Warning: KubeCF and cf-operator versions

KubeCF and cf-operator interoperate closely. Before you deploy a specific version combination, make sure they were confirmed to work. For more information see Section 3.4, “Releases and Associated Versions”.

The supported upgrade method is to install all upgrades, in order. Skipping releases is not supported. This table matches the Helm chart versions to each release as well as other version related information.

The following are required to deploy and use SUSE Cloud Application Platform on SUSE CaaS Platform:

When creating a SUSE CaaS Platform cluster, take note of the following general guidelines to ensure there are sufficient resources available to run a SUSE Cloud Application Platform deployment:

As a minimum, a SUSE Cloud Application Platform deployment with a basic workload will require:

For steps to deploy a SUSE CaaS Platform cluster, refer to the SUSE CaaS Platform Deployment Guide at https://documentation.suse.com/suse-caasp/single-html/caasp-deployment/

Before proceeding with the deployment, take note of the following to ensure the SUSE CaaS Platform cluster is suitable for a deployment of SUSE Cloud Application Platform:

Helm is a Kubernetes package manager used to install and manage SUSE Cloud Application Platform. This requires installing the Helm client, helm, on your remote management workstation. Cloud Application Platform requires Helm 3. For more information regarding Helm, refer to the documentation at https://helm.sh/docs/.

Warning

Make sure that you are installing and using Helm 3 and not Helm 2.

If your remote management workstation has the SUSE CaaS Platform package repository, install helm by running

tux > sudo zypper install helm3
tux > sudo update-alternatives --set helm /usr/bin/helm3

Otherwise, helm can be installed by referring to the documentation at https://helm.sh/docs/intro/install/.

In some SUSE Cloud Application Platform instance groups, such as bits, database and singleton-blobstore require a storage class. To learn more about storage classes, see https://kubernetes.io/docs/concepts/storage/storage-classes/. Examples of provisioners include:

By default, SUSE Cloud Application Platform will use the cluster's default storage class. To designate or change the default storage class, refer to https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/ for instructions.

In some cases, the default and predefined storage classes may not be suitable for certain workloads. If this is the case, operators can define their own custom StorageClass resource according to the specification at https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource.

With the storage class defined, run:

tux > kubectl create --filename my-storage-class.yaml

Then verify the storage class is available by running

tux > kubectl get storageclass

If operators do not want to use the default storage class or one does not exist, a storage class must be specified by setting the kube.storage_class value in your kubecf-config-values.yaml configuration file to the name of the storage class as seen in this example.

kube:
  storage_class: my-storage-class

SUSE Cloud Application Platform is configured using Helm values (see https://helm.sh/docs/chart_template_guide/values_files/ . Helm values can be set as either command line parameters or using a values.yaml file. The following values.yaml file, called kubecf-config-values.yaml in this guide, provides an example of a SUSE Cloud Application Platform configuration.

Warning: kubecf-config-values.yaml changes

The format of the kubecf-config-values.yaml file has been restructured completely in Cloud Application Platform 2.x. Do not re-use the Cloud Application Platform 1.x version of the file. Instead, see the default file in the appendix in Section A.1, “Complete suse/kubecf values.yaml File” and pick parameters according to your needs.

Ensure system_domain maps to the load balancer configured for your SUSE CaaS Platform cluster (see https://documentation.suse.com/suse-caasp/single-html/caasp-deployment/#loadbalancer).

Warning: Supported Domains

When selecting a domain, SUSE Cloud Application Platform expects system_domain to be either a subdomain or a root domain. Setting system_domain to a top-level domain, such as suse, is not supported.

### Example deployment configuration file
### kubecf-config-values.yaml

system_domain: example.com

credentials:
  cf_admin_password: changeme
  uaa_admin_client_secret: alsochangeme

### This block is required due to the log-cache issue described below
properties:
  log-cache:
    log-cache:
      memory_limit_percent: 3

### This block is required due to the log-cache issue described below
###
### The value for key may need to be replaced depending on
### how notes in your cluster are labeled
###
### The value(s) listed under values may need to be
### replaced depending on how notes in your cluster are labeled
operations:
  inline:
  - type: replace
    path: /instance_groups/name=log-cache/env?/bosh/agent/settings/affinity
    value:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - LABEL_VALUE_OF_NODE

This section describes the process to secure traffic passing through your SUSE Cloud Application Platform deployment. This is achieved by using certificates to set up Transport Layer Security (TLS) for the router component. Providing certificates for the router traffic is optional. In a default deployment, without operator-provided certificates, generated certificates will be used.

settings:
  router:
    tls:
      crt: |
        -----BEGIN CERTIFICATE-----
        MIIEEjCCAfoCCQCWC4NErLzy3jANBgkqhkiG9w0BAQsFADBGMQswCQYDVQQGEwJD
        QTETMBEGA1UECAwKU29tZS1TdGF0ZTEOMAwGA1UECgwFTXlPcmcxEjAQBgNVBAMM
        CU15Q0Euc2l0ZTAeFw0xODA5MDYxNzA1MTRaFw0yMDAxMTkxNzA1MTRaMFAxCzAJ
        ...
        xtNNDwl2rnA+U0Q48uZIPSy6UzSmiNaP3PDR+cOak/mV8s1/7oUXM5ivqkz8pEJo
        M3KrIxZ7+MbdTvDOh8lQplvFTeGgjmUDd587Gs4JsormqOsGwKd1BLzQbGELryV9
        1usMOVbUuL8mSKVvgqhbz7vJlW1+zwmrpMV3qgTMoHoJWGx2n5g=
        -----END CERTIFICATE-----
      key: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEpAIBAAKCAQEAm4JMchGSqbZuqc4LdryJpX2HnarWPOW0hUkm60DL53f6ehPK
        T5Dtb2s+CoDX9A0iTjGZWRD7WwjpiiuXUcyszm8y9bJjP3sIcTnHWSgL/6Bb3KN5
        G5D8GHz7eMYkZBviFvygCqEs1hmfGCVNtgiTbAwgBTNsrmyx2NygnF5uy4KlkgwI
        ...
        GORpbQKBgQDB1/nLPjKxBqJmZ/JymBl6iBnhIgVkuUMuvmqES2nqqMI+r60EAKpX
        M5CD+pq71TuBtbo9hbjy5Buh0+QSIbJaNIOdJxU7idEf200+4anzdaipyCWXdZU+
        MPdJf40awgSWpGdiSv6hoj0AOm+lf4AsH6yAqw/eIHXNzhWLRvnqgA==
        -----END RSA PRIVATE KEY----

This section describes how to use an ingress controller (see https://kubernetes.io/docs/concepts/services-networking/ingress/) to manage access to the services in the cluster. Using an ingress controller is optional. In a default deployment, load balancers are used instead.

Note that only the NGINX Ingress Controller has been verified to be compatible with Cloud Application Platform. Other Ingress controller alternatives may work, but compatibility with Cloud Application Platform is not supported.

Important

This feature requires SUSE Cloud Application Platform 2.0.1 or newer.

Operators can set affinity/anti-affinity rules to restrict how the scheduler determines the placement of a given pod on a given node. This can be achieved through node affinity/anti-affinity, where placement is determined by node labels (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity), or pod affinity/anti-affinity, where pod placement is determined by labels on pods that are already running on the node (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity).

In SUSE Cloud Application Platform, a default configuration will have following affinity/anti-affinity rules already in place:

Note that to ensure an optimal spread of the pods across worker nodes we recommend running 5 or more worker nodes to satisfy both of the default anti-affinity constraints. An operator can also specify custom affinity rules via the sizing.instance-group.affinity helm parameter and any affinity rules specified here will overwrite the default rule, not merge with it.

sizing:
   asactors:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0
   asapi:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0

sizing:
  api:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname
  database:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname

Cloud Foundry Application Runtime (CFAR) uses a blobstore (see https://docs.cloudfoundry.org/concepts/cc-blobstore.html) to store the source code that developers push, stage, and run. This section explains how to configure an external blobstore for the Cloud Controller component of your SUSE Cloud Application Platform deployment. Using an external blobstore is optional. In a default deployment, an internal blobstore is used.

SUSE Cloud Application Platform relies on ops files (see https://github.com/cloudfoundry/cf-deployment/blob/master/operations/README.md) provided by cf-deployment (see https://github.com/cloudfoundry/cf-deployment) releases for external blobstore configurations. The default configuration for the blobstore is singleton.

SUSE Cloud Application Platform can be configured to use an external database system, such as a data service offered by a cloud service provider or an existing high availability database server. In a default deployment, an internal single availability database is used.

To configure your deployment to use an external database, please follow the instructions below.

The current SUSE Cloud Application Platform release is compatible with the following types and versions of external databases:

4.11.1 Configuration #

This section describes how to enable and configure your deployment to connect to an external database. The configuration options are specified through Helm values inside the kubecf-config-values.yaml. The deployment and configuration of the external database itself is the responsibility of the operator and beyond the scope of this documentation. It is assumed the external database has been deployed and accessible.

Important: Configuration during Initial Install Only

Configuration of SUSE Cloud Application Platform to use an external database must be done during the initial installation and cannot be changed afterwards.

All the databases listed in the config snippet below need to exist before installing KubeCF. One way of doing that is manually running CREATE DATABASE IF NOT EXISTS database-name for each database.

The following snippet of the kubecf-config-values.yaml contains an example of an external database configuration.

features:
  embedded_database:
    enabled: false
  external_database:
    enabled: true
    require_ssl: false
    ca_cert: ~
    type: mysql
    host: hostname
    port: 3306
    databases:
      uaa:
        name: uaa
        password: root
        username: root
      cc:
        name: cloud_controller
        password: root
        username: root
      bbs:
        name: diego
        password: root
        username: root
      routing_api:
        name: routing-api
        password: root
        username: root
      policy_server:
        name: network_policy
        password: root
        username: root
      silk_controller:
        name: network_connectivity
        password: root
        username: root
      locket: 
        name: locket
        password: root
        username: root
      credhub:        
        name: credhub
        password: root
        username: root

Download the SUSE Kubernetes charts repository with Helm:

tux > helm repo add suse https://kubernetes-charts.suse.com/

You may replace the example suse name with any name. Verify with helm:

tux > helm repo list
NAME       URL
stable     https://kubernetes-charts.storage.googleapis.com
local      http://127.0.0.1:8879/charts
suse       https://kubernetes-charts.suse.com/

List your chart names, as you will need these for some operations:

tux > helm search repo suse
NAME                            CHART VERSION        APP VERSION    DESCRIPTION
suse/cf-operator                7.2.1+0.gaeb6ef3    2.1.1          A Helm chart for cf-operator, the k8s operator ....
suse/console                    4.4.1                2.1.1          A Helm chart for deploying SUSE Stratos Console
suse/kubecf                     2.7.13                2.1.1          A Helm chart for KubeCF
suse/metrics                    1.3.0                2.1.1          A Helm chart for Stratos Metrics
suse/minibroker                 1.2.0                               A minibroker for your minikube
suse/nginx-ingress              0.28.4               0.15.0         An nginx Ingress controller that uses ConfigMap to store ...
...

SUSE Cloud Application Platform can be integrated with identity providers to help manage authentication of users. Integrating SUSE Cloud Application Platform with other identity providers is optional. In a default deployment, a built-in UAA server (https://docs.cloudfoundry.org/uaa/uaa-overview.html) is used to manage user accounts and authentication.

The Lightweight Directory Access Protocol (LDAP) is an example of an identity provider that Cloud Application Platform integrates with. This section describes the necessary components and steps in order to configure the integration. See User Account and Authentication LDAP Integration for more information.

If the current capacity of your Cloud Application Platform deployment is insufficient for your workloads, you can expand the capacity using the procedure in this section.

These instructions assume you have followed the procedure in Chapter 4, Deploying SUSE Cloud Application Platform on SUSE CaaS Platform and have a running Cloud Application Platform deployment on SUSE® CaaS Platform.

The following are required to deploy and use SUSE Cloud Application Platform on AKS:

Log in to your Azure account, which should have the Contributor role.

tux > az login

You can set up an AKS cluster with an automatically generated service principal. Note that to be be able to create a service principal your user account must have permissions to register an application with your Azure Active Directory tenant, and to assign the application to a role in your subscription. For details, see https://docs.microsoft.com/en-us/azure/aks/kubernetes-service-principal#automatically-create-and-use-a-service-principal.

Alternatively, you can specify an existing service principal but the service principal must have sufficient rights to be able to create resources at the appropriate level, for example resource group, subscription etc. For more details please see:

Specify the following additional parameters for creating the cluster: node count, a username for SSH access to the nodes, SSH key, VM type, VM disk size and optionally, the Kubernetes version and a nodepool name.

tux > az aks create --resource-group my-resource-group --name cap-aks \
 --node-count 3 --admin-username cap-user \
 --ssh-key-value /path/to/some_key.pub --node-vm-size Standard_DS4_v2 \
 --node-osdisk-size 100 --nodepool-name mypool

For more az aks create options see https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-create.

This takes a few minutes. When it is completed, fetch your kubectl credentials. The default behavior for az aks get-credentials is to merge the new credentials with the existing default configuration, and to set the new credentials as as the current Kubernetes context. The context name is your AKS_NAME value. You should first backup your current configuration, or move it to a different location, then fetch the new credentials:

tux > az aks get-credentials --resource-group $RG_NAME --name $AKS_NAME
 Merged "cap-aks" as current context in /home/tux/.kube/config

Verify that you can connect to your cluster:

tux > kubectl get nodes

When all nodes are in a ready state and all pods are running, proceed to the next steps.

Helm is a Kubernetes package manager used to install and manage SUSE Cloud Application Platform. This requires installing the Helm client, helm, on your remote management workstation. Cloud Application Platform requires Helm 3. For more information regarding Helm, refer to the documentation at https://helm.sh/docs/.

Warning

Make sure that you are installing and using Helm 3 and not Helm 2.

If your remote management workstation has the SUSE CaaS Platform package repository, install helm by running

tux > sudo zypper install helm3
tux > sudo update-alternatives --set helm /usr/bin/helm3

Otherwise, helm can be installed by referring to the documentation at https://helm.sh/docs/intro/install/.

In some SUSE Cloud Application Platform instance groups, such as bits, database, diego-cell, and singleton-blobstore require a storage class for persistent data. To learn more about storage classes, see https://kubernetes.io/docs/concepts/storage/storage-classes/.

By default, SUSE Cloud Application Platform will use the cluster's default storage class. To designate or change the default storage class, refer to https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/ for instructions.

In some cases, the default and predefined storage classes may not be suitable for certain workloads. If this is the case, operators can define their own custom StorageClass resource according to the specification at https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource.

With the storage class defined, run:

tux > kubectl create --filename my-storage-class.yaml

Then verify the storage class is available by running

tux > kubectl get storageclass

If operators do no want to use the default storage class or one does not exist, a storage class must be specified by setting the kube.storage_class value in your kubecf-config-values.yaml configuration file to the name of the storage class as seen in this example.

kube:
  storage_class: my-storage-class

The following file, kubecf-config-values.yaml, provides a minimal example deployment configuration.

Warning: kubecf-config-values.yaml changes

The format of the kubecf-config-values.yaml file has been restructured completely in Cloud Application Platform 2.x. Do not re-use the Cloud Application Platform 1.x version of the file. Instead, see the default file in the appendix in Section A.1, “Complete suse/kubecf values.yaml File” and pick parameters according to your needs.

Warning: Supported Domains

When selecting a domain, SUSE Cloud Application Platform expects system_domain to be either a subdomain or a root domain. Setting system_domain to a top-level domain, such as suse, is not supported.

### Example deployment configuration file
### kubecf-config-values.yaml

system_domain: example.com

credentials:
  cf_admin_password: changeme
  uaa_admin_client_secret: alsochangeme

### This block is required due to the log-cache issue described below
properties:
  log-cache:
    log-cache:
      memory_limit_percent: 3

### This block is required due to the log-cache issue described below
###
### The value for key may need to be replaced depending on
### how notes in your cluster are labeled
###
### The value(s) listed under values may need to be
### replaced depending on how notes in your cluster are labeled
operations:
  inline:
  - type: replace
    path: /instance_groups/name=log-cache/env?/bosh/agent/settings/affinity
    value:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - LABEL_VALUE_OF_NODE

This section describes the process to secure traffic passing through your SUSE Cloud Application Platform deployment. This is achieved by using certificates to set up Transport Layer Security (TLS) for the router component. Providing certificates for the router traffic is optional. In a default deployment, without operator-provided certificates, generated certificates will be used.

settings:
  router:
    tls:
      crt: |
        -----BEGIN CERTIFICATE-----
        MIIEEjCCAfoCCQCWC4NErLzy3jANBgkqhkiG9w0BAQsFADBGMQswCQYDVQQGEwJD
        QTETMBEGA1UECAwKU29tZS1TdGF0ZTEOMAwGA1UECgwFTXlPcmcxEjAQBgNVBAMM
        CU15Q0Euc2l0ZTAeFw0xODA5MDYxNzA1MTRaFw0yMDAxMTkxNzA1MTRaMFAxCzAJ
        ...
        xtNNDwl2rnA+U0Q48uZIPSy6UzSmiNaP3PDR+cOak/mV8s1/7oUXM5ivqkz8pEJo
        M3KrIxZ7+MbdTvDOh8lQplvFTeGgjmUDd587Gs4JsormqOsGwKd1BLzQbGELryV9
        1usMOVbUuL8mSKVvgqhbz7vJlW1+zwmrpMV3qgTMoHoJWGx2n5g=
        -----END CERTIFICATE-----
      key: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEpAIBAAKCAQEAm4JMchGSqbZuqc4LdryJpX2HnarWPOW0hUkm60DL53f6ehPK
        T5Dtb2s+CoDX9A0iTjGZWRD7WwjpiiuXUcyszm8y9bJjP3sIcTnHWSgL/6Bb3KN5
        G5D8GHz7eMYkZBviFvygCqEs1hmfGCVNtgiTbAwgBTNsrmyx2NygnF5uy4KlkgwI
        ...
        GORpbQKBgQDB1/nLPjKxBqJmZ/JymBl6iBnhIgVkuUMuvmqES2nqqMI+r60EAKpX
        M5CD+pq71TuBtbo9hbjy5Buh0+QSIbJaNIOdJxU7idEf200+4anzdaipyCWXdZU+
        MPdJf40awgSWpGdiSv6hoj0AOm+lf4AsH6yAqw/eIHXNzhWLRvnqgA==
        -----END RSA PRIVATE KEY----

This section describes how to use an ingress controller (see https://kubernetes.io/docs/concepts/services-networking/ingress/) to manage access to the services in the cluster. Using an ingress controller is optional. In a default deployment, load balancers are used instead.

Note that only the NGINX Ingress Controller has been verified to be compatible with Cloud Application Platform. Other Ingress controller alternatives may work, but compatibility with Cloud Application Platform is not supported.

Important

This feature requires SUSE Cloud Application Platform 2.0.1 or newer.

Operators can set affinity/anti-affinity rules to restrict how the scheduler determines the placement of a given pod on a given node. This can be achieved through node affinity/anti-affinity, where placement is determined by node labels (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity), or pod affinity/anti-affinity, where pod placement is determined by labels on pods that are already running on the node (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity).

In SUSE Cloud Application Platform, a default configuration will have following affinity/anti-affinity rules already in place:

Note that to ensure an optimal spread of the pods across worker nodes we recommend running 5 or more worker nodes to satisfy both of the default anti-affinity constraints. An operator can also specify custom affinity rules via the sizing.instance-group.affinity helm parameter and any affinity rules specified here will overwrite the default rule, not merge with it.

sizing:
   asactors:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0
   asapi:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0

sizing:
  api:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname
  database:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname

Cloud Foundry Application Runtime (CFAR) uses a blobstore (see https://docs.cloudfoundry.org/concepts/cc-blobstore.html) to store the source code that developers push, stage, and run. This section explains how to configure an external blobstore for the Cloud Controller component of your SUSE Cloud Application Platform deployment. Using an external blobstore is optional. In a default deployment, an internal blobstore is used.

SUSE Cloud Application Platform relies on ops files (see https://github.com/cloudfoundry/cf-deployment/blob/master/operations/README.md) provided by cf-deployment (see https://github.com/cloudfoundry/cf-deployment) releases for external blobstore configurations. The default configuration for the blobstore is singleton.

SUSE Cloud Application Platform can be configured to use an external database system, such as a data service offered by a cloud service provider or an existing high availability database server. In a default deployment, an internal single availability database is used.

To configure your deployment to use an external database, please follow the instructions below.

The current SUSE Cloud Application Platform release is compatible with the following types and versions of external databases:

5.11.1 Configuration #

This section describes how to enable and configure your deployment to connect to an external database. The configuration options are specified through Helm values inside the kubecf-config-values.yaml. The deployment and configuration of the external database itself is the responsibility of the operator and beyond the scope of this documentation. It is assumed the external database has been deployed and accessible.

Important: Configuration during Initial Install Only

Configuration of SUSE Cloud Application Platform to use an external database must be done during the initial installation and cannot be changed afterwards.

All the databases listed in the config snippet below need to exist before installing KubeCF. One way of doing that is manually running CREATE DATABASE IF NOT EXISTS database-name for each database.

The following snippet of the kubecf-config-values.yaml contains an example of an external database configuration.

features:
  embedded_database:
    enabled: false
  external_database:
    enabled: true
    require_ssl: false
    ca_cert: ~
    type: mysql
    host: hostname
    port: 3306
    databases:
      uaa:
        name: uaa
        password: root
        username: root
      cc:
        name: cloud_controller
        password: root
        username: root
      bbs:
        name: diego
        password: root
        username: root
      routing_api:
        name: routing-api
        password: root
        username: root
      policy_server:
        name: network_policy
        password: root
        username: root
      silk_controller:
        name: network_connectivity
        password: root
        username: root
      locket: 
        name: locket
        password: root
        username: root
      credhub:        
        name: credhub
        password: root
        username: root

Download the SUSE Kubernetes charts repository with Helm:

tux > helm repo add suse https://kubernetes-charts.suse.com/

You may replace the example suse name with any name. Verify with helm:

tux > helm repo list
NAME       URL
stable     https://kubernetes-charts.storage.googleapis.com
local      http://127.0.0.1:8879/charts
suse       https://kubernetes-charts.suse.com/

List your chart names, as you will need these for some operations:

tux > helm search repo suse
NAME                            CHART VERSION        APP VERSION    DESCRIPTION
suse/cf-operator                7.2.1+0.gaeb6ef3    2.1.1          A Helm chart for cf-operator, the k8s operator ....
suse/console                    4.4.1                2.1.1          A Helm chart for deploying SUSE Stratos Console
suse/kubecf                     2.7.13                2.1.1          A Helm chart for KubeCF
suse/metrics                    1.3.0                2.1.1          A Helm chart for Stratos Metrics
suse/minibroker                 1.2.0                               A minibroker for your minikube
suse/nginx-ingress              0.28.4               0.15.0         An nginx Ingress controller that uses ConfigMap to store ...
...

This section describes how to deploy SUSE Cloud Application Platform with a Azure Standard SKU load balancer.

Warning: KubeCF and cf-operator versions

KubeCF and cf-operator interoperate closely. Before you deploy a specific version combination, make sure they were confirmed to work. For more information see Section 3.4, “Releases and Associated Versions”.

SUSE Cloud Application Platform can be integrated with identity providers to help manage authentication of users. Integrating SUSE Cloud Application Platform with other identity providers is optional. In a default deployment, a built-in UAA server (https://docs.cloudfoundry.org/uaa/uaa-overview.html) is used to manage user accounts and authentication.

The Lightweight Directory Access Protocol (LDAP) is an example of an identity provider that Cloud Application Platform integrates with. This section describes the necessary components and steps in order to configure the integration. See User Account and Authentication LDAP Integration for more information.

If the current capacity of your Cloud Application Platform deployment is insufficient for your workloads, you can expand the capacity using the procedure in this section.

These instructions assume you have followed the procedure in Chapter 5, Deploying SUSE Cloud Application Platform on Microsoft Azure Kubernetes Service (AKS) and have a running Cloud Application Platform deployment on Microsoft AKS. The instructions below will use environment variables defined in Section 5.2, “Create Resource Group and AKS Instance”.

The following are required to deploy and use SUSE Cloud Application Platform on EKS:

Now you can create an EKS cluster using eksctl. Be sure to keep in mind the following minimum requirements of the cluster.

As a minimal example, the following command will create an EKS cluster. To see additional configuration parameters, see eksctl create cluster --help.

tux > eksctl create cluster --name kubecf --version 1.14 \
--nodegroup-name standard-workers --node-type t3.xlarge \
--nodes 3 --node-volume-size 100 \
--region us-east-2 --managed \
--ssh-access --ssh-public-key /path/to/some_key.pub

Helm is a Kubernetes package manager used to install and manage SUSE Cloud Application Platform. This requires installing the Helm client, helm, on your remote management workstation. Cloud Application Platform requires Helm 3. For more information regarding Helm, refer to the documentation at https://helm.sh/docs/.

Warning

Make sure that you are installing and using Helm 3 and not Helm 2.

If your remote management workstation has the SUSE CaaS Platform package repository, install helm by running

tux > sudo zypper install helm3
tux > sudo update-alternatives --set helm /usr/bin/helm3

Otherwise, helm can be installed by referring to the documentation at https://helm.sh/docs/intro/install/.

In some SUSE Cloud Application Platform instance groups, such as bits, database, diego-cell, and singleton-blobstore require a storage class for persistent data. To learn more about storage classes, see https://kubernetes.io/docs/concepts/storage/storage-classes/.

By default, SUSE Cloud Application Platform will use the cluster's default storage class. To designate or change the default storage class, refer to https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/ for instructions.

In some cases, the default and predefined storage classes may not be suitable for certain workloads. If this is the case, operators can define their own custom StorageClass resource according to the specification at https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource.

With the storage class defined, run:

tux > kubectl create --filename my-storage-class.yaml

Then verify the storage class is available by running

tux > kubectl get storageclass

If operators do no want to use the default storage class or one does not exist, a storage class must be specified by setting the kube.storage_class value in your kubecf-config-values.yaml configuration file to the name of the storage class as seen in this example.

kube:
  storage_class: my-storage-class

Use this example kubecf-config-values.yaml as a template for your configuration.

Warning: kubecf-config-values.yaml changes

The format of the kubecf-config-values.yaml file has been restructured completely in Cloud Application Platform 2.x. Do not re-use the Cloud Application Platform 1.x version of the file. Instead, see the default file in the appendix in Section A.1, “Complete suse/kubecf values.yaml File” and pick parameters according to your needs.

Warning: Supported Domains

When selecting a domain, SUSE Cloud Application Platform expects system_domain to be either a subdomain or a root domain. Setting system_domain to a top-level domain, such as suse, is not supported.

### Example deployment configuration file
### kubecf-config-values.yaml

system_domain: example.com

credentials:
  cf_admin_password: changeme
  uaa_admin_client_secret: alsochangeme

### This block is required due to the log-cache issue described below
properties:
  log-cache:
    log-cache:
      memory_limit_percent: 3

### This block is required due to the log-cache issue described below
###
### The value for key may need to be replaced depending on
### how notes in your cluster are labeled
###
### The value(s) listed under values may need to be
### replaced depending on how notes in your cluster are labeled
operations:
  inline:
  - type: replace
    path: /instance_groups/name=log-cache/env?/bosh/agent/settings/affinity
    value:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - LABEL_VALUE_OF_NODE

This section describes the process to secure traffic passing through your SUSE Cloud Application Platform deployment. This is achieved by using certificates to set up Transport Layer Security (TLS) for the router component. Providing certificates for the router traffic is optional. In a default deployment, without operator-provided certificates, generated certificates will be used.

settings:
  router:
    tls:
      crt: |
        -----BEGIN CERTIFICATE-----
        MIIEEjCCAfoCCQCWC4NErLzy3jANBgkqhkiG9w0BAQsFADBGMQswCQYDVQQGEwJD
        QTETMBEGA1UECAwKU29tZS1TdGF0ZTEOMAwGA1UECgwFTXlPcmcxEjAQBgNVBAMM
        CU15Q0Euc2l0ZTAeFw0xODA5MDYxNzA1MTRaFw0yMDAxMTkxNzA1MTRaMFAxCzAJ
        ...
        xtNNDwl2rnA+U0Q48uZIPSy6UzSmiNaP3PDR+cOak/mV8s1/7oUXM5ivqkz8pEJo
        M3KrIxZ7+MbdTvDOh8lQplvFTeGgjmUDd587Gs4JsormqOsGwKd1BLzQbGELryV9
        1usMOVbUuL8mSKVvgqhbz7vJlW1+zwmrpMV3qgTMoHoJWGx2n5g=
        -----END CERTIFICATE-----
      key: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEpAIBAAKCAQEAm4JMchGSqbZuqc4LdryJpX2HnarWPOW0hUkm60DL53f6ehPK
        T5Dtb2s+CoDX9A0iTjGZWRD7WwjpiiuXUcyszm8y9bJjP3sIcTnHWSgL/6Bb3KN5
        G5D8GHz7eMYkZBviFvygCqEs1hmfGCVNtgiTbAwgBTNsrmyx2NygnF5uy4KlkgwI
        ...
        GORpbQKBgQDB1/nLPjKxBqJmZ/JymBl6iBnhIgVkuUMuvmqES2nqqMI+r60EAKpX
        M5CD+pq71TuBtbo9hbjy5Buh0+QSIbJaNIOdJxU7idEf200+4anzdaipyCWXdZU+
        MPdJf40awgSWpGdiSv6hoj0AOm+lf4AsH6yAqw/eIHXNzhWLRvnqgA==
        -----END RSA PRIVATE KEY----

This section describes how to use an ingress controller (see https://kubernetes.io/docs/concepts/services-networking/ingress/) to manage access to the services in the cluster. Using an ingress controller is optional. In a default deployment, load balancers are used instead.

Note that only the NGINX Ingress Controller has been verified to be compatible with Cloud Application Platform. Other Ingress controller alternatives may work, but compatibility with Cloud Application Platform is not supported.

Important

This feature requires SUSE Cloud Application Platform 2.0.1 or newer.

Operators can set affinity/anti-affinity rules to restrict how the scheduler determines the placement of a given pod on a given node. This can be achieved through node affinity/anti-affinity, where placement is determined by node labels (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity), or pod affinity/anti-affinity, where pod placement is determined by labels on pods that are already running on the node (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity).

In SUSE Cloud Application Platform, a default configuration will have following affinity/anti-affinity rules already in place:

Note that to ensure an optimal spread of the pods across worker nodes we recommend running 5 or more worker nodes to satisfy both of the default anti-affinity constraints. An operator can also specify custom affinity rules via the sizing.instance-group.affinity helm parameter and any affinity rules specified here will overwrite the default rule, not merge with it.

sizing:
   asactors:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0
   asapi:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0

sizing:
  api:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname
  database:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname

Cloud Foundry Application Runtime (CFAR) uses a blobstore (see https://docs.cloudfoundry.org/concepts/cc-blobstore.html) to store the source code that developers push, stage, and run. This section explains how to configure an external blobstore for the Cloud Controller component of your SUSE Cloud Application Platform deployment. Using an external blobstore is optional. In a default deployment, an internal blobstore is used.

SUSE Cloud Application Platform relies on ops files (see https://github.com/cloudfoundry/cf-deployment/blob/master/operations/README.md) provided by cf-deployment (see https://github.com/cloudfoundry/cf-deployment) releases for external blobstore configurations. The default configuration for the blobstore is singleton.

SUSE Cloud Application Platform can be configured to use an external database system, such as a data service offered by a cloud service provider or an existing high availability database server. In a default deployment, an internal single availability database is used.

To configure your deployment to use an external database, please follow the instructions below.

The current SUSE Cloud Application Platform release is compatible with the following types and versions of external databases:

6.11.1 Configuration #

This section describes how to enable and configure your deployment to connect to an external database. The configuration options are specified through Helm values inside the kubecf-config-values.yaml. The deployment and configuration of the external database itself is the responsibility of the operator and beyond the scope of this documentation. It is assumed the external database has been deployed and accessible.

Important: Configuration during Initial Install Only

Configuration of SUSE Cloud Application Platform to use an external database must be done during the initial installation and cannot be changed afterwards.

All the databases listed in the config snippet below need to exist before installing KubeCF. One way of doing that is manually running CREATE DATABASE IF NOT EXISTS database-name for each database.

The following snippet of the kubecf-config-values.yaml contains an example of an external database configuration.

features:
  embedded_database:
    enabled: false
  external_database:
    enabled: true
    require_ssl: false
    ca_cert: ~
    type: mysql
    host: hostname
    port: 3306
    databases:
      uaa:
        name: uaa
        password: root
        username: root
      cc:
        name: cloud_controller
        password: root
        username: root
      bbs:
        name: diego
        password: root
        username: root
      routing_api:
        name: routing-api
        password: root
        username: root
      policy_server:
        name: network_policy
        password: root
        username: root
      silk_controller:
        name: network_connectivity
        password: root
        username: root
      locket: 
        name: locket
        password: root
        username: root
      credhub:        
        name: credhub
        password: root
        username: root

Download the SUSE Kubernetes charts repository with Helm:

tux > helm repo add suse https://kubernetes-charts.suse.com/

You may replace the example suse name with any name. Verify with helm:

tux > helm repo list
NAME            URL
stable          https://kubernetes-charts.storage.googleapis.com
local           http://127.0.0.1:8879/charts
suse            https://kubernetes-charts.suse.com/

List your chart names, as you will need these for some operations:

tux > helm search repo suse
NAME                            CHART VERSION        APP VERSION    DESCRIPTION
suse/cf-operator                7.2.1+0.gaeb6ef3    2.1.1          A Helm chart for cf-operator, the k8s operator ....
suse/console                    4.4.1                2.1.1          A Helm chart for deploying SUSE Stratos Console
suse/kubecf                     2.7.13                2.1.1          A Helm chart for KubeCF
suse/metrics                    1.3.0                2.1.1          A Helm chart for Stratos Metrics
suse/minibroker                 1.2.0                               A minibroker for your minikube
suse/nginx-ingress              0.28.4               0.15.0         An nginx Ingress controller that uses ConfigMap to store ...
...

This section describes how to deploy SUSE Cloud Application Platform on Amazon EKS.

Warning: KubeCF and cf-operator versions

KubeCF and cf-operator interoperate closely. Before you deploy a specific version combination, make sure they were confirmed to work. For more information see Section 3.4, “Releases and Associated Versions”.

SUSE Cloud Application Platform can be integrated with identity providers to help manage authentication of users. Integrating SUSE Cloud Application Platform with other identity providers is optional. In a default deployment, a built-in UAA server (https://docs.cloudfoundry.org/uaa/uaa-overview.html) is used to manage user accounts and authentication.

The Lightweight Directory Access Protocol (LDAP) is an example of an identity provider that Cloud Application Platform integrates with. This section describes the necessary components and steps in order to configure the integration. See User Account and Authentication LDAP Integration for more information.

If the current capacity of your Cloud Application Platform deployment is insufficient for your workloads, you can expand the capacity using the procedure in this section.

These instructions assume you have followed the procedure in Chapter 6, Deploying SUSE Cloud Application Platform on Amazon Elastic Kubernetes Service (EKS) and have a running Cloud Application Platform deployment on Amazon EKS.

The following are required to deploy and use SUSE Cloud Application Platform on GKE:

In order to deploy SUSE Cloud Application Platform, create a cluster that:

Get the kubeconfig file for your cluster.

tux > gcloud container clusters get-credentials --zone ${CLUSTER_ZONE:?required} ${CLUSTER_NAME:?required} --project example-project

Helm is a Kubernetes package manager used to install and manage SUSE Cloud Application Platform. This requires installing the Helm client, helm, on your remote management workstation. Cloud Application Platform requires Helm 3. For more information regarding Helm, refer to the documentation at https://helm.sh/docs/.

Warning

Make sure that you are installing and using Helm 3 and not Helm 2.

If your remote management workstation has the SUSE CaaS Platform package repository, install helm by running

tux > sudo zypper install helm3
tux > sudo update-alternatives --set helm /usr/bin/helm3

Otherwise, helm can be installed by referring to the documentation at https://helm.sh/docs/intro/install/.

In some SUSE Cloud Application Platform instance groups, such as bits, database, diego-cell, and singleton-blobstore require a storage class for persistent data. To learn more about storage classes, see https://kubernetes.io/docs/concepts/storage/storage-classes/.

By default, SUSE Cloud Application Platform will use the cluster's default storage class. To designate or change the default storage class, refer to https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/ for instructions.

In some cases, the default and predefined storage classes may not be suitable for certain workloads. If this is the case, operators can define their own custom StorageClass resource according to the specification at https://kubernetes.io/docs/concepts/storage/storage-classes/#the-storageclass-resource.

With the storage class defined, run:

tux > kubectl create --filename my-storage-class.yaml

Then verify the storage class is available by running

tux > kubectl get storageclass

If operators do no want to use the default storage class or one does not exist, a storage class must be specified by setting the kube.storage_class value in your kubecf-config-values.yaml configuration file to the name of the storage class as seen in this example.

kube:
  storage_class: my-storage-class

The following file, kubecf-config-values.yaml, provides a minimal example deployment configuration.

Warning: kubecf-config-values.yaml changes

The format of the kubecf-config-values.yaml file has been restructured completely in Cloud Application Platform 2.x. Do not re-use the Cloud Application Platform 1.x version of the file. Instead, see the default file in the appendix in Section A.1, “Complete suse/kubecf values.yaml File” and pick parameters according to your needs.

Warning: Supported Domains

When selecting a domain, SUSE Cloud Application Platform expects system_domain to be either a subdomain or a root domain. Setting system_domain to a top-level domain, such as suse, is not supported.

### Example deployment configuration file
### kubecf-config-values.yaml

system_domain: example.com

credentials:
  cf_admin_password: changeme
  uaa_admin_client_secret: alsochangeme

### This block is required due to the log-cache issue described below
properties:
  log-cache:
    log-cache:
      memory_limit_percent: 3

### This block is required due to the log-cache issue described below
###
### The value for key may need to be replaced depending on
### how notes in your cluster are labeled
###
### The value(s) listed under values may need to be
### replaced depending on how notes in your cluster are labeled
operations:
  inline:
  - type: replace
    path: /instance_groups/name=log-cache/env?/bosh/agent/settings/affinity
    value:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - LABEL_VALUE_OF_NODE

This section describes the process to secure traffic passing through your SUSE Cloud Application Platform deployment. This is achieved by using certificates to set up Transport Layer Security (TLS) for the router component. Providing certificates for the router traffic is optional. In a default deployment, without operator-provided certificates, generated certificates will be used.

settings:
  router:
    tls:
      crt: |
        -----BEGIN CERTIFICATE-----
        MIIEEjCCAfoCCQCWC4NErLzy3jANBgkqhkiG9w0BAQsFADBGMQswCQYDVQQGEwJD
        QTETMBEGA1UECAwKU29tZS1TdGF0ZTEOMAwGA1UECgwFTXlPcmcxEjAQBgNVBAMM
        CU15Q0Euc2l0ZTAeFw0xODA5MDYxNzA1MTRaFw0yMDAxMTkxNzA1MTRaMFAxCzAJ
        ...
        xtNNDwl2rnA+U0Q48uZIPSy6UzSmiNaP3PDR+cOak/mV8s1/7oUXM5ivqkz8pEJo
        M3KrIxZ7+MbdTvDOh8lQplvFTeGgjmUDd587Gs4JsormqOsGwKd1BLzQbGELryV9
        1usMOVbUuL8mSKVvgqhbz7vJlW1+zwmrpMV3qgTMoHoJWGx2n5g=
        -----END CERTIFICATE-----
      key: |
        -----BEGIN RSA PRIVATE KEY-----
        MIIEpAIBAAKCAQEAm4JMchGSqbZuqc4LdryJpX2HnarWPOW0hUkm60DL53f6ehPK
        T5Dtb2s+CoDX9A0iTjGZWRD7WwjpiiuXUcyszm8y9bJjP3sIcTnHWSgL/6Bb3KN5
        G5D8GHz7eMYkZBviFvygCqEs1hmfGCVNtgiTbAwgBTNsrmyx2NygnF5uy4KlkgwI
        ...
        GORpbQKBgQDB1/nLPjKxBqJmZ/JymBl6iBnhIgVkuUMuvmqES2nqqMI+r60EAKpX
        M5CD+pq71TuBtbo9hbjy5Buh0+QSIbJaNIOdJxU7idEf200+4anzdaipyCWXdZU+
        MPdJf40awgSWpGdiSv6hoj0AOm+lf4AsH6yAqw/eIHXNzhWLRvnqgA==
        -----END RSA PRIVATE KEY----

This section describes how to use an ingress controller (see https://kubernetes.io/docs/concepts/services-networking/ingress/) to manage access to the services in the cluster. Using an ingress controller is optional. In a default deployment, load balancers are used instead.

Note that only the NGINX Ingress Controller has been verified to be compatible with Cloud Application Platform. Other Ingress controller alternatives may work, but compatibility with Cloud Application Platform is not supported.

Important

This feature requires SUSE Cloud Application Platform 2.0.1 or newer.

Operators can set affinity/anti-affinity rules to restrict how the scheduler determines the placement of a given pod on a given node. This can be achieved through node affinity/anti-affinity, where placement is determined by node labels (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity), or pod affinity/anti-affinity, where pod placement is determined by labels on pods that are already running on the node (see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity).

In SUSE Cloud Application Platform, a default configuration will have following affinity/anti-affinity rules already in place:

Note that to ensure an optimal spread of the pods across worker nodes we recommend running 5 or more worker nodes to satisfy both of the default anti-affinity constraints. An operator can also specify custom affinity rules via the sizing.instance-group.affinity helm parameter and any affinity rules specified here will overwrite the default rule, not merge with it.

sizing:
   asactors:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0
   asapi:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: topology.kubernetes.io/zone
               operator: In
               values:
               - 0

sizing:
  api:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname
  database:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: quarks.cloudfoundry.org/quarks-statefulset-name
                operator: In
                values:
                - sample_group
            topologyKey: kubernetes.io/hostname

Cloud Foundry Application Runtime (CFAR) uses a blobstore (see https://docs.cloudfoundry.org/concepts/cc-blobstore.html) to store the source code that developers push, stage, and run. This section explains how to configure an external blobstore for the Cloud Controller component of your SUSE Cloud Application Platform deployment. Using an external blobstore is optional. In a default deployment, an internal blobstore is used.

SUSE Cloud Application Platform relies on ops files (see https://github.com/cloudfoundry/cf-deployment/blob/master/operations/README.md) provided by cf-deployment (see https://github.com/cloudfoundry/cf-deployment) releases for external blobstore configurations. The default configuration for the blobstore is singleton.

SUSE Cloud Application Platform can be configured to use an external database system, such as a data service offered by a cloud service provider or an existing high availability database server. In a default deployment, an internal single availability database is used.

To configure your deployment to use an external database, please follow the instructions below.

The current SUSE Cloud Application Platform release is compatible with the following types and versions of external databases:

7.12.1 Configuration #

This section describes how to enable and configure your deployment to connect to an external database. The configuration options are specified through Helm values inside the kubecf-config-values.yaml. The deployment and configuration of the external database itself is the responsibility of the operator and beyond the scope of this documentation. It is assumed the external database has been deployed and accessible.

Important: Configuration during Initial Install Only

Configuration of SUSE Cloud Application Platform to use an external database must be done during the initial installation and cannot be changed afterwards.

All the databases listed in the config snippet below need to exist before installing KubeCF. One way of doing that is manually running CREATE DATABASE IF NOT EXISTS database-name for each database.

The following snippet of the kubecf-config-values.yaml contains an example of an external database configuration.

features:
  embedded_database:
    enabled: false
  external_database:
    enabled: true
    require_ssl: false
    ca_cert: ~
    type: mysql
    host: hostname
    port: 3306
    databases:
      uaa:
        name: uaa
        password: root
        username: root
      cc:
        name: cloud_controller
        password: root
        username: root
      bbs:
        name: diego
        password: root
        username: root
      routing_api:
        name: routing-api
        password: root
        username: root
      policy_server:
        name: network_policy
        password: root
        username: root
      silk_controller:
        name: network_connectivity
        password: root
        username: root
      locket: 
        name: locket
        password: root
        username: root
      credhub:        
        name: credhub
        password: root
        username: root

Download the SUSE Kubernetes charts repository with Helm:

tux > helm repo add suse https://kubernetes-charts.suse.com/

You may replace the example suse name with any name. Verify with helm:

tux > helm repo list
NAME       URL
stable     https://kubernetes-charts.storage.googleapis.com
local      http://127.0.0.1:8879/charts
suse       https://kubernetes-charts.suse.com/

List your chart names, as you will need these for some operations:

tux > helm search repo suse
NAME                            CHART VERSION        APP VERSION    DESCRIPTION
suse/cf-operator                7.2.1+0.gaeb6ef3    2.1.1          A Helm chart for cf-operator, the k8s operator ....
suse/console                    4.4.1                2.1.1          A Helm chart for deploying SUSE Stratos Console
suse/kubecf                     2.7.13                2.1.1          A Helm chart for KubeCF
suse/metrics                    1.3.0                2.1.1          A Helm chart for Stratos Metrics
suse/minibroker                 1.2.0                               A minibroker for your minikube
suse/nginx-ingress              0.28.4               0.15.0         An nginx Ingress controller that uses ConfigMap to store ...
...

This section describes how to deploy SUSE Cloud Application Platform on Google GKE, and how to configure your DNS records.

Warning: KubeCF and cf-operator versions

KubeCF and cf-operator interoperate closely. Before you deploy a specific version combination, make sure they were confirmed to work. For more information see Section 3.4, “Releases and Associated Versions”.

SUSE Cloud Application Platform can be integrated with identity providers to help manage authentication of users. Integrating SUSE Cloud Application Platform with other identity providers is optional. In a default deployment, a built-in UAA server (https://docs.cloudfoundry.org/uaa/uaa-overview.html) is used to manage user accounts and authentication.

The Lightweight Directory Access Protocol (LDAP) is an example of an identity provider that Cloud Application Platform integrates with. This section describes the necessary components and steps in order to configure the integration. See User Account and Authentication LDAP Integration for more information.

If the current capacity of your Cloud Application Platform deployment is insufficient for your workloads, you can expand the capacity using the procedure in this section.

These instructions assume you have followed the procedure in Chapter 7, Deploying SUSE Cloud Application Platform on Google Kubernetes Engine (GKE) and have a running Cloud Application Platform deployment on Microsoft AKS. The instructions below will use environment variables defined in Section 7.2, “Creating a GKE cluster”.

The steps in this section describe how to install Stratos on SUSE® CaaS Platform without an external load balancer, instead mapping a worker node to your SUSE Cloud Application Platform domain as described in Section 4.5, “Deployment Configuration”. These instructions assume you have followed the procedure in Chapter 4, Deploying SUSE Cloud Application Platform on SUSE CaaS Platform, have deployed kubecf successfully, and have created a default storage class.

If you are using SUSE Enterprise Storage as your storage back-end, copy the secret into the Stratos namespace:

tux > kubectl get secret ceph-secret-admin --output json --namespace default | \
sed 's/"namespace": "default"/"namespace": "stratos"/' | kubectl create --filename -

You should already have the Stratos charts when you downloaded the SUSE charts repository (see Section 4.12, “Add the Kubernetes Charts Repository”). Search your Helm repository to verify that you have the suse/console chart:

tux > helm search repo suse
NAME                            CHART VERSION        APP VERSION    DESCRIPTION
suse/cf-operator                7.2.1+0.gaeb6ef3    2.1.1          A Helm chart for cf-operator, the k8s operator ....
suse/console                    4.4.1                2.1.1          A Helm chart for deploying SUSE Stratos Console
suse/kubecf                     2.7.13                2.1.1          A Helm chart for KubeCF
suse/metrics                    1.3.0                2.1.1          A Helm chart for Stratos Metrics
suse/minibroker                 1.2.0                               A minibroker for your minikube
suse/nginx-ingress              0.28.4               0.15.0         An nginx Ingress controller that uses ConfigMap to store ...
...

Create a YAML file, called stratos-config-values.yaml in this example, and use it to make configurations to the Stratos Helm chart.

### example Stratos deployment configuration file
### stratos-config-values.yaml

console:
  # Use local admin user instead of UAA
  localAdminPassword: changeme

Note: Technology Preview Features

Some Stratos releases may include features as part of a technology preview. Technology preview features are for evaluation purposes only and not supported for production use. To see the technology preview features available for a given release, refer to https://github.com/SUSE/stratos/blob/master/CHANGELOG.md.

To enable technology preview features, add the console.techPreview Helm value to your stratos-config-values.yaml and set it to true.

### example Stratos deployment configuration file                        
### stratos-config-values.yaml
                                                                                 
console:
  techPreview: true

Create a namespace for your Stratos deployment.

tux > kubectl create namespace stratos

Deploy Stratos using Helm.

tux > helm install susecf-console suse/console \
--namespace stratos \
--values stratos-config-values.yaml

You can monitor the status of your stratos deployment with the watch command:

tux > watch --color 'kubectl get pods --namespace stratos'

When stratos is successfully deployed, the following is observed:

Press Ctrl–C to exit the watch command.

When the stratos deployment completes, query with Helm to view your release information:

tux > helm status susecf-console
LAST DEPLOYED: Wed Mar 27 06:51:36 2019
NAMESPACE: stratos
STATUS: DEPLOYED

RESOURCES:
==> v1/Secret
NAME                           TYPE    DATA  AGE
susecf-console-secret          Opaque  2     3h
susecf-console-mariadb-secret  Opaque  2     3h

==> v1/PersistentVolumeClaim
NAME                                  STATUS  VOLUME                                    CAPACITY  ACCESSMODES  STORAGECLASS  AGE
susecf-console-upgrade-volume         Bound   pvc-711380d4-5097-11e9-89eb-fa163e15acf0  20Mi      RWO          persistent    3h
susecf-console-encryption-key-volume  Bound   pvc-711b5275-5097-11e9-89eb-fa163e15acf0  20Mi      RWO          persistent    3h
console-mariadb                       Bound   pvc-7122200c-5097-11e9-89eb-fa163e15acf0  1Gi       RWO          persistent    3h

==> v1/Service
NAME                    CLUSTER-IP      EXTERNAL-IP                                                PORT(S)   AGE
susecf-console-mariadb  172.24.137.195  <none>                                                     3306/TCP  3h
susecf-console-ui-ext   172.24.80.22    10.86.101.115,172.28.0.31,172.28.0.36,172.28.0.7,172.28.0.22  8443/TCP  3h

==> v1beta1/Deployment
NAME        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
stratos-db  1        1        1           1          3h

==> v1beta1/StatefulSet
NAME     DESIRED  CURRENT  AGE
stratos  1        1        3h

Find the external IP address with kubectl get service susecf-console-ui-ext --namespace stratos to access your new Stratos Web console, for example https://10.86.101.115:8443, or use the domain you created for it, and its port, for example https://example.com:8443. Proceed past the warnings about the self-signed certificates and log in as admin with the password you created in stratos-config-values.yaml

Figure 8.1: Stratos UI Cloud Foundry Console #

8.1.1 Connecting SUSE® CaaS Platform to Stratos #

Stratos can show information from your SUSE® CaaS Platform environment.

To enable this, you must register and connect your SUSE® CaaS Platform environment with Stratos.

In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view - you should be shown the "Register new Endpoint" view.

1. In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view.

2. On the Register a new Endpoint view, click the SUSE CaaS Platform button.

3. Enter a memorable name for your SUSE® CaaS Platform environment in the Name field. For example, my-endpoint.

4. Enter the URL of the API server for your Kubernetes environment in the Endpoint Address field. Run kubectl cluster-info and use the value of Kubernetes master as the URL.

   tux > kubectl cluster-info

5. Activate the Skip SSL validation for the endpoint check box if using self-signed certificates.

6. Click Register.

7. Activate the Connect to my-endpoint now (optional). check box.

8. Provide a valid kubeconfig file for your SUSE® CaaS Platform environment.

9. Click Connect.

10. In the Stratos UI, go to Kubernetes in the left-hand side navigation. Information for your SUSE® CaaS Platform environment should now be displayed as in the following figure.

Figure 8.2: Kubernetes Environment Information on Stratos #

Before deploying Stratos, ensure kubecf has been successfully deployed on Amazon EKS (see Chapter 6, Deploying SUSE Cloud Application Platform on Amazon Elastic Kubernetes Service (EKS)).

Configure a scoped storage class for your Stratos deployment. Create a configuration file, called scoped-storage-class.yaml in this example, using the following as a template. Specify the region you are using as the zone and be sure to include the letter (for example, the letter a in us-west-2a) identifier to indicate the Availability Zone used:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gp2scoped
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
  zone: "us-west-2a"
reclaimPolicy: Retain
mountOptions:
  - debug

Create the storage class using the scoped-storage-class.yaml configuration file:

tux > kubectl create --filename scoped-storage-class.yaml

Verify the storage class has been created:

tux > kubectl get storageclass
NAME            PROVISIONER             AGE
gp2 (default)   kubernetes.io/aws-ebs   1d
gp2scoped       kubernetes.io/aws-ebs   1d

Create a YAML file, called stratos-config-values.yaml in this example, and use it to make configurations to the Stratos Helm chart.

### example Stratos deployment configuration file
### stratos-config-values.yaml

console:
  # Use local admin user instead of UAA
  localAdminPassword: changeme

services:
  loadbalanced: true

kube:
  storage_class:
    persistent: gp2scoped

Note: Technology Preview Features

Some Stratos releases may include features as part of a technology preview. Technology preview features are for evaluation purposes only and not supported for production use. To see the technology preview features available for a given release, refer to https://github.com/SUSE/stratos/blob/master/CHANGELOG.md.

To enable technology preview features, add the console.techPreview Helm value to your stratos-config-values.yaml and set it to true.

### example Stratos deployment configuration file                        
### stratos-config-values.yaml
                                                                                 
console:
  techPreview: true

Create a namespace for your Stratos deployment.

tux > kubectl create namespace stratos

Deploy Stratos using Helm.

tux > helm install susecf-console suse/console
\
--namespace stratos \
--values stratos-config-values.yaml

You can monitor the status of your stratos deployment with the watch command:

tux > watch --color 'kubectl get pods --namespace stratos'

When stratos is successfully deployed, the following is observed:

Press Ctrl–C to exit the watch command.

Obtain the host name of the service exposed through the public load balancer:

tux > kubectl get service susecf-console-ui-ext --namespace stratos

Use this host name to create a CNAME record. After the record is created, access the console in a web browser by navigating to the domain mapped to the host name of the service retrieved from the kubectl get service step. Upon successfully logging in, you should see something similar to the following figure.

Figure 8.3: Stratos UI Cloud Foundry Console #

8.2.1 Connecting Amazon EKS to Stratos #

Stratos can show information from your Amazon EKS environment.

To enable this, you must register and connect your Amazon EKS environment with Stratos.

In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view - you should be shown the "Register new Endpoint" view.

1. In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view.

2. On the Register a new Endpoint view, click the Amazon EKS button.

3. Enter a memorable name for your Amazon EKS environment in the Name field. For example, my-endpoint.

4. Enter the URL of the API server for your Kubernetes environment in the Endpoint Address field. Run kubectl cluster-info and use the value of Kubernetes master as the URL.

   tux > kubectl cluster-info

5. Activate the Skip SSL validation for the endpoint check box if using self-signed certificates.

6. Click Register.

7. Activate the Connect to my-endpoint now (optional). check box.

8. Enter the name of your Amazon EKS cluster in the Cluster field.

9. Enter your AWS Access Key ID in the Access Key ID field.

10. Enter your AWS Secret Access Key in the Secret Access Key field.

11. Click Connect.

12. In the Stratos UI, go to Kubernetes in the left-hand side navigation. Information for your Amazon EKS environment should now be displayed as in the following figure.

Figure 8.4: Kubernetes Environment Information on Stratos #

Before deploying Stratos, ensure kubecf has been successfully deployed on Microsoft AKS (see Chapter 5, Deploying SUSE Cloud Application Platform on Microsoft Azure Kubernetes Service (AKS)).

Create a YAML file, called stratos-config-values.yaml in this example, and use it to make configurations to the Stratos Helm chart.

### example Stratos deployment configuration file
### stratos-config-values.yaml

console:
  # Use local admin user instead of UAA
  localAdminPassword: changeme

services:
  loadbalanced: true

Note: Technology Preview Features

Some Stratos releases may include features as part of a technology preview. Technology preview features are for evaluation purposes only and not supported for production use. To see the technology preview features available for a given release, refer to https://github.com/SUSE/stratos/blob/master/CHANGELOG.md.

To enable technology preview features, add the console.techPreview Helm value to your stratos-config-values.yaml and set it to true.

### example Stratos deployment configuration file                        
### stratos-config-values.yaml
                                                                                 
console:
  techPreview: true

Create a namespace for your Stratos deployment.

tux > kubectl create namespace stratos

Deploy Stratos using Helm.

tux > helm install susecf-console suse/console
\
--namespace stratos \
--values stratos-config-values.yaml

You can monitor the status of your stratos deployment with the watch command:

tux > watch --color 'kubectl get pods --namespace stratos'

When stratos is successfully deployed, the following is observed:

Press Ctrl–C to exit the watch command.

Obtain the IP address of the service exposed through the public load balancer:

tux > kubectl get service susecf-console-ui-ext --namespace stratos

Use this IP address to create an A record. After the record is created, access the console in a web browser by navigating to the domain mapped to the IP address of the service retrieved from the kubectl get service step. Upon successfully logging in, you should see something similar to the following figure.

Figure 8.5: Stratos UI Cloud Foundry Console #

8.3.1 Connecting Microsoft AKS to Stratos #

Stratos can show information from your Microsoft AKS environment.

To enable this, you must register and connect your Microsoft AKS environment with Stratos.

In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view - you should be shown the "Register new Endpoint" view.

1. In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view.

2. On the Register a new Endpoint view, click the Azure AKS button.

3. Enter a memorable name for your Microsoft AKS environment in the Name field. For example, my-endpoint.

4. Enter the URL of the API server for your Kubernetes environment in the Endpoint Address field. Run kubectl cluster-info and use the value of Kubernetes master as the URL.

   tux > kubectl cluster-info

5. Activate the Skip SSL validation for the endpoint check box if using self-signed certificates.

6. Click Register.

7. Activate the Connect to my-endpoint now (optional). check box.

8. Provide a valid kubeconfig file for your Microsoft AKS environment.

9. Click Connect.

10. In the Stratos UI, go to Kubernetes in the left-hand side navigation. Information for your Microsoft AKS environment should now be displayed as in the following figure.

Figure 8.6: Kubernetes Environment Information on Stratos #

Before deploying Stratos, ensure kubecf has been successfully deployed on Google GKE (see Chapter 7, Deploying SUSE Cloud Application Platform on Google Kubernetes Engine (GKE)).

Create a YAML file, called stratos-config-values.yaml in this example, and use it to make configurations to the Stratos Helm chart.

### example Stratos deployment configuration file
### stratos-config-values.yaml

console:
  # Use local admin user instead of UAA
  localAdminPassword: changeme

services:
  loadbalanced: true

Note: Technology Preview Features

Some Stratos releases may include features as part of a technology preview. Technology preview features are for evaluation purposes only and not supported for production use. To see the technology preview features available for a given release, refer to https://github.com/SUSE/stratos/blob/master/CHANGELOG.md.

To enable technology preview features, add the console.techPreview Helm value to your stratos-config-values.yaml and set it to true.

### example Stratos deployment configuration file                        
### stratos-config-values.yaml
                                                                                 
console:
  techPreview: true

Create a namespace for your Stratos deployment.

tux > kubectl create namespace stratos

Deploy Stratos using Helm.

tux > helm install susecf-console suse/console
\
--namespace stratos \
--values stratos-config-values.yaml

You can monitor the status of your stratos deployment with the watch command:

tux > watch --color 'kubectl get pods --namespace stratos'

When stratos is successfully deployed, the following is observed:

Press Ctrl–C to exit the watch command.

Obtain the IP address of the service exposed through the public load balancer:

tux > kubectl get service susecf-console-ui-ext --namespace stratos

Use this IP address to create an A record. After the record is created, access the console in a web browser by navigating to the domain mapped to the IP address of the service retrieved from the kubectl get service step. Upon successfully logging in, you should see something similar to the following figure.

Figure 8.7: Stratos UI Cloud Foundry Console #

8.4.1 Connecting Google GKE to Stratos #

Stratos can show information from your Google GKE environment.

To enable this, you must register and connect your Google GKE environment with Stratos.

In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view - you should be shown the "Register new Endpoint" view.

1. In the Stratos UI, go to Endpoints in the left-hand side navigation and click on the + icon in the top-right of the view.

2. On the Register a new Endpoint view, click the Google Kubernetes Engine button.

3. Enter a memorable name for your Microsoft AKS environment in the Name field. For example, my-endpoint.

4. Enter the URL of the API server for your Kubernetes environment in the Endpoint Address field. Run kubectl cluster-info and use the value of Kubernetes master as the URL.

   tux > kubectl cluster-info

5. Activate the Skip SSL validation for the endpoint check box if using self-signed certificates.

6. Click Register.

7. Activate the Connect to my-endpoint now (optional). check box.

8. Provide a valid Application Default Credentials file for your Google GKE environment. Generate the file using the command below. The command saves the credentials to a file named application_default_credentials.json and outputs the path of the file.

   tux > gcloud auth application-default login

9. Click Connect.

10. In the Stratos UI, go to Kubernetes in the left-hand side navigation. Information for your Google GKE environment should now be displayed as in the following figure.

Figure 8.8: Kubernetes Environment Information on Stratos #

For instructions to upgrade Stratos, follow the process described in Chapter 13, Upgrading SUSE Cloud Application Platform. Take note that kubecf is upgraded prior to upgrading Stratos.

Stratos Metrics provides a Helm chart for deploying Prometheus (see https://prometheus.io/) and the following metrics exporters to Kubernetes:

The Stratos Metrics Helm chart deploys a Prometheus server and the configured Exporters and fronts the Prometheus server with an nginx server to provide authenticated access to Prometheus (currently basic authentication over HTTPS).

When required by configuration, it also contains an initialization script that will setup users in the UAA that have correct scopes/permissions to be able to read data from the Cloud Foundry Firehose and/or API.

Lastly, the Helm chart generates a small metadata file in the root of the nginx server that is used by Stratos to determine which Cloud Foundry and Kubernetes clusters the Prometheus server is providing Metrics for.

To learn more about Stratos Metrics and its full list of configuration options, see https://github.com/SUSE/stratos-metrics.

firehoseExporter:
  instances: 1

cloudFoundry:
  apiEndpoint: https://api.example.com
  uaaAdminClient: admin
  uaaAdminClientSecret: password
  skipSslVerification: "true"
env:
  DOPPLER_PORT: 443
kubernetes:
  apiEndpoint: kube_server_address.example.com
metrics:
  username: username
  password: password
prometheus:
  kubeStateMetrics:
    enabled: true
  server:
    storageClass: "persistent"
services:
  loadbalanced: true

tux > kubectl get secret ceph-secret-admin --output json --namespace default | \
sed 's/"namespace": "default"/"namespace": "metrics"/' | kubectl create --filename -

tux > kubectl create namespace metrics

tux > helm install susecf-metrics suse/metrics \
--namespace metrics \
--values kubecf-config-values.yaml \
--values stratos-metrics-values.yaml

$ watch --color 'kubectl get pods --namespace metrics'

When using Eirini, it is important to take into consideration:

The following prerequisites are required:

All the Cloud Application Platform Helm charts include an imagelist.txt file that lists all images from the registry.suse.com registry under the cap organization. They can be mirrored to a local registry with the following script.

Replace the value of MIRROR with your registry's domain.

#!/bin/bash

MIRROR=MY_REGISTRY.COM

set -ex

function mirror {
    CHART=$1
    CHARTDIR=$(mktemp -d)
    helm fetch suse/$1 --untar --untardir=${CHARTDIR}
    IMAGES=$(cat ${CHARTDIR}/**/imagelist.txt)
    for IMAGE in ${IMAGES}; do
        echo $IMAGE
        docker pull registry.suse.com/cap/$IMAGE
        docker tag registry.suse.com/cap/$IMAGE $MIRROR/cap/$IMAGE
        docker push $MIRROR/cap/$IMAGE
    done
    docker save -o ${CHART}-images.tar.gz \
           $(perl -E "say qq(registry.suse.com/cap/\$_) for @ARGV" ${IMAGES})
    rm -r ${CHARTDIR}
}

mirror cf-operator
mirror kubecf
mirror console
mirror metrics
mirror minibroker

The script above will both mirror to a local registry and save the images in a local tarball that can be restored with docker load foo-images.tgz. In general only one of these mechanisms will be needed.

Also take note of the following regarding the script provided above.

Before deploying Cloud Application Platform using helm install, ensure the following in your kubecf-config-values.yaml has been updated to point to your registry, and not registry.suse.com.

kube:
  registry:
    # example registry domain
    hostname: "MY_REGISTRY.COM"
    username: ""
    password: ""
  organization: "cap"