Applies to SUSE OpenStack Cloud Crowbar 9

3 Generate SUSE OpenStack Cloud Self Signed Certificate #

The purpose of this document is to help set up SSL Support for several services in SUSE OpenStack Cloud. The scope of this document covers all public endpoints in your OpenStack cluster. In most cases you want to have a Secure CA or External CA where your certificates are signed. You will sign with either a public CA or self signed CA, and include x509 extensions for Subject Alt Names since there might be a highly available control plane with alternate names.

3.1 Create the CA Root Pair #

This section demonstrates how to create the certificate on the crowbar or admin node of the SUSE OpenStack Cloud Cluster.

Note

To avoid external access to your CA Root Pair, put it on an air-gapped system that is permanently isolated from the internet and unplug any cables from the ethernet port.

Procedure 3.1: Prepare the directory structure #

Create a directory for your CA Root pair:

           # ssh root@crowbar
           # mkdir -p ~/ssl/root/ca

Create a directory structure and add index.txt and serial files to act as flat database of all signed certificates:

           # cd ~/ssl/root/ca
           # mkdir certs crl newcerts private csr
           # chmod 700 private
           # touch index.txt
           # echo 1000 > serial

Procedure 3.2: Prepare the configuration file #

This procedure takes you through the full set up. Note that when you setup the crowbar server, there is a structure already setup under /etc/ssl. This is where SUSE Linux typically contains the CA cert bundle created through YaST when the SMT server is set up. However, if you are using an external SMT server you will not have this.

Copy /etc/ssl/openssl.cnf file to your setup. We can use this since it is completely annotated.
```
           # cp /etc/ssl/openssl.cnf ./
```
Edit the file and change the location variable:
```
           dir = /root/ssl/root/ca
           # Where everything is kept
```
Note
Make sure dir is the directory where we created /root/ssl/root/ca.

Procedure 3.3: Create the root key #

Create the root key encrypted with AES 256-bit encryption and a password, using 4096 bits for the creation.
```
          # cd ~/ssl/root/ca
          # openssl genrsa -aes256 -out private/cakey.pem 4096
```
You will be asked to enter a password here and then verify it.
```
          # chmod 400 private/cakey.pem
```

Procedure 3.4: Create the root certificates #

Use the root key (cakey.pem) to create the root certificate (cacert.pem). Its important to give it a long expiration since all the certificates signed from it will become invalid when it expires.

           # cd ~/ssl/root/ca
           # openssl req -config openssl.cnf -key private/cakey.pem -new -x509 -days 10950 -sha256 -extensions v3_ca -out cacert.pem
           Enter pass phrase for cakey.pem: enteryourpassword
           You are about to be asked to enter information that will be incorporated
           into your certificate request.
           -----
           Country Name (2 letter code) [AU]:US
           State or Province Name []:Idaho
           Locality Name []:Meridian
           Organization Name []:SUSEDojo
           Organizational Unit Name []:dojo
           Common Name []:susedojo.com
           Email Address []:admin@susedojo.com

           # chmod 444 cacert.pem

Procedure 3.5: Verify the root certificates #

Verify the certificate has the correct dates of validity and the algorithm used, Issuer, Subject, and x509v3 extensions. The issuer and subject are the same since it is self signed.
```
          # cd ~/ssl/root/ca
          # openssl x509 -noout -text -in cacert.pem
```

3.2 Sign server and client certificates #

This section is if you are the perspective certificate authority (CA).

Procedure 3.6: Prepare config file #

Modify the penssl.cnf config file and add a line to the [ v3_req ] section:

          # cd ~/ssl/root/ca
          # vi openssl.cnf
          find v3_req
          Add the following line:
          subjectAltName = DNS:public.your_server_name.your_domain.com, DNS: cluster-control.your_domain.com
          At the bottom of the file create section server_cert with the follwing:
          [ server_cert ]
          subjectAltName = subjectAltName = DNS:public.your_server_name.your_domain.com, DNS: cluster-control.your_domain.com

The first DNS name would be used if you only have a single node controller as you need the public URL for that server in your cluster. For example, public.db8-ae-ed-77-14-9e.susedojo.com. If you have a haproxy setup for your cluster or pacemaker, you have a cluster URL. For example, you may have public.cluster.your_domain.com and you need to have cluster.your_domain.com and public.cluster.your_domain.com as Alternative DNS names. This public URL can be used for all endpoints unless you have multiple High Availability Clusters for your control plane.
Save and close the file after you have those entered correctly.

Procedure 3.7: Create a key #

Create a key minus the -aes256 option so that you are not presented with a password each time you restart a service. (i.e. Apache service) also in 2048 bit so it is quicker to decrypt.
```
           # cd ~/ssl/root/ca
           # openssl genrsa -out private/susedojo-com.key.pem 2048
           # chmod 400 private/susedojo-com.key.pem
```

Procedure 3.8: Create a certificate #

Use the private key we just created to create a certificate signing request (CSR). The common name must be a fully qualified domain name (i.e. www.susedojo.com) The Organization Name must be the same as the Organization Name in the CA.

            # cd ~/ssl/root/ca
            # openssl req -config openssl.cnf -key private/susedojo-com.key.pem -new -sha256 -out csr/susedojo-com.csr.pem
            You are about to be asked to enter information that will be incorporated
            into your certificate request.
            -----
            Country Name (2 letter code) [XX]:US
            State or Province Name []:Idaho
            Locality Name []:Meridian
            Organization Name []:SUSEDojo
            Organizational Unit Name []:dojo
            Common Name []:susedojo.com
            Email Address []:admin@susedojo.com

Note

You may be prompted for a challenge password and company name. This can be left blank.

Create the certificate using the CA to sign the CSR, using the server_cert extension as this will be used on a server. We will give it one year of validity.

            # cd ~/ssl/root/ca
            # openssl ca -config openssl.cnf -extensions server_cert -days 365 -notext -md sha256 -in  csr/susedojo-com.csr.pem -out certs/susedojo-com.cert.pem
              Using configuration from openssl.cnf
              Enter pass phrase for /root/ssl/root/ca/private/cakey.pem:
              Check that the request matches the signature
              Signature ok
                      Serial Number: 4096 (0x1000)
                      Validity
                        Not Before: Aug  8 04:21:08 2018 GMT
                        Not After: Aug  8 04:21:08 2019 GMT
                     Subject:
                          countryName               = US
                          stateOrProvinceName       = Idaho
                          organizationName          = SUSEDojo
                          organizationalUnitName    = dojo
                          commonName                = susedojo.com
                          emailAddress              = admin@susedojo.com
                     X509v3 extensions:
                         X509v3 Basic Constraints:
                            CA:FALSE
                        X509v3 Key Usage:
                              Digital Signature, Non Repudiation, Key Encipherment
                         X509v3 Subject Alternative Name:
                             DNS:public.db8-ae-ed-77-14-9e.susedojo.com
            Certificate is to be certified until Aug  8 04:21:08 2019 GMT (365 days)
            Sign the certificate? [y/n]:y

            1 out of 1 certificate requests certified, commit? [y/n]y
            Write out database with 1 new entries
            Data Base Updated

            # chmod 444 certs/susedojo-com.cert.pem

The index.txt file should now contain a line referring to the new certificate that has been created. For example, the output should look like the following:

            V       190808042108Z           1000    unknown
            /C=US/ST=Idaho/O=SUSEDojo/OU=dojo/CN=susedojo.com/emailAddress=admin@susedojo.com

Procedure 3.9: Verifying the certificate #

Enter the following in your terminal:

             # openssl x509 -noout -text -in certs/susedojo-com.cert.pem

You will notice the Issuer is the CA and you can also see the Subject Alternative Name as well in the extensions section.

             # openssl verify -CAfile cacert.pem certs/susedojo-com.cert.pem
             certs/susedojo-com.cert.pem: OK

3.3 Deploying the certificate #

Now you are ready to copy the newly created certificate and key to the control node or controllers in the cluster.
```
         # scp newcerts/1000.pem control:/root/
         # scp private/susedojo-com.key control:/root/
```

Copy them into the right location on the controller host:

         # cp susedojo-com.key.pem /etc/keystone/ssl/private
         # cp 1000.pem /etc/keystone/ssl/certs
         # cd /etc/keystone/ssl/certs
         # mv signing_cert.pem signing_cert.pem.todays_date
         # cp 1000.pem signing_cert.pem
         # cd /etc/keystone/ssl/private
         # old signing_key.pem
         # cp susedojo-com.key.pem signing_key.pem

Rerun the Barclamp for keystone in order to apply this change to the cluster.

3.4 Generate Public Certificate using Let’s Encrypt #

Let’s Encrypt is a free, automated, and open Certificate Authority. Its Root is trusted by all major operating systems now. For SUSE Linux Enterprise Server 12 SP3 and higher, the ISRG Root X1 is available in /etc/ssl/certs/ISRG_Root_X1.pem. If not, apply the latest updates for your operating system.

Let’s Encrypt has several clients to choose from depending on your needs. For this example, we will be using the acme.sh client, which is written in bash and gives us greater flexibility and ease in our solution.

The next steps walk you through the installation of acme.sh and the issue of a certificate with Let’s Encrypt followed by the automated load of the certificate in OpenStack for the various API endpoints available.

Procedure 3.10: Installation of acme.sh letsencrypt client #

Create a new directory for letsencrypt and clone the acme.sh repo:

         # mkdir letsencrypt
         # cd letsencrypt
         # git clone https://github.com/Neilpang/acme.sh.git
         # cd acme.sh

The system is prepared for installing acme.sh.

Install socat:

         # export BRANCH=2 #this makes sure you are using the v2 api version of letsencrypt
         # zypper in -y socat

Install acme.sh:
```
         # ./acme.sh --install
```
After the install of acme.sh is finished, you should see a new directory /root/.acme.sh/ where acme.sh lives and all of its environment, account info, and certificates are stored. We recommend using this as a backup location if you are using a backup tool.

Procedure 3.11: Issue a wildcard SSL Certificate #

OpenStack and wildcard SSL uses the DNS validation method by validating your domain using a TXT record that can either be added manually or using the many (over 3 dozen) available DNS API’s.

Note

It is important to a wildcard certificate as you have the ability to use the same one for all of your public API endpoints in the OpenStack Cloud environment. Additional Cloud Native services like Kubernetes can also take advantage of it.

The manual DNS mode is a method that displays the DNS records that have to be created in your DNS servers. It is beneficial to automate the injection of DNS records as the maximum days a certificate is viable is 60 days. In order to issue your wildcard certificate, the command without optional settings is:
```
        # acme.sh --issue -d yourdomain.com -d *.yourdomain.com --dns
```

To debug or test, add the following optional settings:

        # acme.sh --debug --test –issue -d yourdomain.com -d *.yourdomain.com --dns

A message returns. For example:

        Add the following TXT record:
        Domain: '_acme-challenge.yourdomain.com'
        TXT value: 'KZvgq3MpOCjUNW7Uzz5nE5kkFdplNk66WGfxE9-H63k'
        Please be aware that you prepend _acme-challenge. before your domain
        so the resulting subdomain will be: _acme-challenge.yourdomain.com

Using this information, you are ready to insert this TXT record into your DNS.
Note
When setting this up for SUSE OpenStack Cloud with Crowbar, you need to have your external DNS server appended to /etc/resolv.conf in order to resolve as crowbar has its own internal DNS. It is not enough to change it in the barclamp as you need the DNS server entry to be at the top of the list in resolv.conf. Crowbar returns to the default after a period of time. Keep in mind that if you want to automate this step every 90 days then you need to ensure the resolv.conf changes every time to bypass the local crowbar DNS Server.

In order to set up TXT record in bind DNS, edit the zone file so it looks like the following example:

        yourdomain.com.     IN NS           admin.yourdomain.com.
        _acme-challenge.yourdomain.com. IN TXT "xxxx...your TXT value string here"

Restart your named services for bind.
Issue the acme-challenge verification of the previous step with the following command:
```
        # acme.sh --renew -d yourdomain.com
```

If the DNS validation is okay, acme.sh issues a wildcard certificate and displays the certificate and private-key path. For example:

        Your cert is in:  /root/.acme.sh/susedojo.com/susedojo.com.cer  
        Your cert key is in:  /root/.acme.sh/susedojo.com/susedojo.com.key  
        v2 chain.
        The intermediate CA cert is in:  /root/.acme.sh/susedojo.com/ca.cer  
        And the full chain certs is in:  /root/.acme.sh/susedojo.com/fullchain.cer_on_issue_success

Notice the location of your certificate and key. These are now ready to be used by OpenStack Cloud.
To automate the process of setting up the TXT record in your DNS servers and prepare it for automated validation, the file account.conf holds account information for the DNS Provider. After exporting the authentication variables, it stores them automatically after the command is executed for later use. To issue your wildcard certificate, the command without optional settings is:
```
        # export LUA_Key=”your_API_token_from_account”
        # export LUA_Email=”cameron@yourdomain.com”
        # acme.sh -d yourdomain.com -d *.yourdomain.com --dns dns_lua
```

You can now view your DNS records and you will see a new TXT record available. When it is finished and the DNS validation is okay, acme.sh issue your wildcard certificate and displays your certificate and private-key path just as before.

        Your cert is in:  /root/.acme.sh/susedojo.com/susedojo.com.cer  
        Your cert key is in:  /root/.acme.sh/susedojo.com/susedojo.com.key  
        v2 chain.
        The intermediate CA cert is in:  /root/.acme.sh/susedojo.com/ca.cer  
        And the full chain certs is in:  /root/.acme.sh/susedojo.com/fullchain.cer_on_issue_success

Procedure 3.12: Set Up Certificate Store on Control and Compute Nodes #

Create a shared location for all Certificates on the control nodes. Execute these commands on all control nodes and compute nodes:
```
        mkdir -p /etc/cloud/ssl/certs
        mkdir -p /etc/cloud/ssl/private
```

Copy all certificates to their shared locations on the control nodes and compute nodes:

        # scp /root/.acme.sh/susedojo.com/susedojo.com.cer \ root@control:/etc/cloud/ssl/certs
        # scp /root/.acme.sh/susedojo.com/ca.cer root@control:/etc/cloud/ssl/certs
        # scp /root/.acme.sh/susedojo.com/fullchain.cer root@control:/etc/cloud/ssl/certs
        # scp /root/.acme.sh/susedojo.com/susedojo.com.key \ root@control:/etc/cloud/ssl/private

Procedure 3.13: Set Up Issued Certificates in Crowbar Barclamps #

Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false.
Figure 3.1: Database Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false.
Figure 3.2: RabbitMQ Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false.
Figure 3.3: Keystone Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false, and Require Client Certificates is true.
Figure 3.4: Glance Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false, and Require Client Certificates is false.
Figure 3.5: Cinder Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false, and Require Client Certificates is false.
Figure 3.6: Neutron Barclamp #
Click Apply. Your changes will apply in a few minutes.
Set your Certificate File and Key File to the proper location, and set the CA Certificates File to the /etc/ssl/ca-bundle.pem in the distribution. Make sure Generate (self-signed) certificates is false, and Certificate is insecure is false, and Require Client Certificates is false.
Figure 3.7: Nova Barclamp #
Click Apply. Your changes will apply in a few minutes.

Each Crowbar barclamp that have SSL support are the same. You can change the same settings and apply your certificate to the remaining barclamps.

Note

Once this is completed, we recommend automating this process as the Let's Encrypt certificates expire after 90 days.