41 Configuring Transport Layer Security (TLS) #

For this example certificate configuration, let us assume there is no FQDN for the external VIP and that you are going to use the default IP address provided by SUSE OpenStack Cloud 9. Let's also assume that for the internal VIP you will use the defaults as well. If you were to call your certificate authority "example-CA," the CA certificate would then be called "example-CA.crt" and the key would be called "example-CA.key." In the following examples, the external VIP certificate will be named "example-public-cert" and the internal VIP certificate will be named "example-internal-cert."

Note

Cautions:

Any time you make a cert change when using your own CA:

Important

Be sure to install your own certificate for all production clouds after installing and testing your cloud. If you ever want to test or troubleshoot later, you will be able to revert to the sample certificate to get back to a stable state for testing.

Note

Unless this is a new deployment, do not update both the certificate and the CA together. Add the CA first and then run a site deploy. Then update the certificate and run tls-reconfigure, FND-CLU-stop, FND-CLU-start and then ardana-reconfigure. If a playbook has failed, rerun it with -vv to get detailed error information. The configure, HAproxy restart, and reconfigure steps are included below. If this is a new deployment and you are adding your own certs/CA before running site.yml this caveat does not apply.

You can add your own certificate by following the instructions below. All changes must go into the file ~/openstack/my_cloud/definition/data/network_groups.yml.

Below are the entries for TLS for the internal and admin load balancers:

- provider: ip-cluster
        name: lb
        tls-components:
        - default
        components:
        # These services do not currently support TLS so they are not listed
        # under tls-components
        - nova-metadata
        roles:
        - internal
        - admin
        cert-file: openstack-internal-cert
        # The openstack-internal-cert is a reserved name and
        # this certificate will be autogenerated. You
        # can bring in your own certificate with a different name

        # cert-file: customer-provided-internal-cert
        # replace this with name of file in "config/tls/certs/"

The configuration processor will also create a request template for each named certificate under info/cert_reqs/ This will be of the form:

info/cert_reqs/customer-provided-internal-cert

These request templates contain the subject Alt-names that the certificates need. You can add to this template before generating your certificate signing request .

You would then send the CSR to your CA to be signed, and once you receive the certificate, place it in config/tls/certs.

When you bring in your own certificate, you may want to bring in the trust chains (or CA certificate) for this certificate. This is usually not required if the CA is a public signer that is typically bundled with the operating system. However, we suggest you include it anyway by copying the file into the directory config/cacerts/.

SUSE OpenStack Cloud generates its own internal certificates but is designed to allow you to bring in your own certificates for the VIPs. Here is the general process.

Edit the load balancer configuration in ~/openstack/my_cloud/definition/data/network_groups.yml:

load-balancers:
 - provider: ip-cluster
 name: lb
 tls-components:
 - default
 components:
 - nova-metadata
 roles:
 - internal
 - admin
 cert-file: example-internal-cert #<<<---- Certificate name for the internal VIP

- provider: ip-cluster
 name: extlb
 external-name: myardana.test #<<<--- Use just IP for the external VIP in this example
 tls-components:
 - default
 roles:
 - public
 cert-file: example-public-cert #<<<---- Certificate name for the external VIP

Commit your changes to the local git repository and run the configuration processor:

ardana > cd ~/openstack/ardana/ansible
ardana > git add -A
ardana > git commit -m "changed VIP certificates"
ardana > ansible-playbook -i hosts/localhost config-processor-run.yml

Verify that certificate requests have been generated by the configuration processor for every certificate file configured in the networks_groups.yml file. In this example, there are two files, as shown from the list command:

ardana > ls ~/openstack/my_cloud/info/cert_reqs
example-internal-cert
example-public-cert

Note

In a production setting you will not perform this step. You will use your company's CA or a valid public CA.

This section demonstrates to how you can create your own self-signed CA and then use this CA to sign server certificates. This CA can be your organization's IT internal CA that is self-signed and whose CA certificates are deployed on your organization's machines. This way the server certificate becomes legitimate.

Note

Please use a unique CN for your example Certificate Authority and do not install multiple CA certificates with the same CN into your cloud.

Copy the commands below to the command line and execute. This will cause the two files, example-CA.key and example-CA.crt to be created:

ardana > export EXAMPLE_CA_KEY_FILE='example-CA.key'
ardana > export EXAMPLE_CA_CERT_FILE='example-CA.crt'
ardana > openssl req -x509 -batch -newkey rsa:2048 -nodes -out "${EXAMPLE_CA_CERT_FILE}" \
-keyout "${EXAMPLE_CA_KEY_FILE}" \
-subj "/C=UK/O=hp/CN=YourOwnUniqueCertAuthorityName" \
-days 365

You can tweak the subj and days settings above to meet your needs, or to test. For instance, if you want to test what happens when a CA expires, you can set 'days' to a very low value. Grab the configuration processor-generated request file from info/cert_reqs/:

ardana > cat ~/openstack/my_cloud/info/cert_reqs/example-internal-cert

Now, copy this file to your working directory and append a .req extension to it.

ardana > cp ~/openstack/my_cloud/info/cert_reqs/example-internal-cert \
example-internal-cert.req

[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[ req_distinguished_name ]
CN = "openstack-vip"

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = "deployerincloud-ccp-c0-m1-mgmt"
DNS.2 = "deployerincloud-ccp-vip-CEI-API-mgmt"
DNS.3 = "deployerincloud-ccp-vip-CND-API-mgmt"
DNS.4 = "deployerincloud-ccp-vip-DES-API-mgmt"
DNS.5 = "deployerincloud-ccp-vip-FND-MDB-mgmt"
DNS.6 = "deployerincloud-ccp-vip-FND-RMQ-mgmt"
DNS.7 = "deployerincloud-ccp-vip-FND-VDB-mgmt"
DNS.8 = "deployerincloud-ccp-vip-FRE-API-mgmt"
DNS.9 = "deployerincloud-ccp-vip-GLA-API-mgmt"
DNS.10 = "deployerincloud-ccp-vip-GLA-REG-mgmt"
DNS.11 = "deployerincloud-ccp-vip-HEA-ACF-mgmt"
DNS.12 = "deployerincloud-ccp-vip-HEA-ACW-mgmt"
DNS.13 = "deployerincloud-ccp-vip-HEA-API-mgmt"
DNS.14 = "deployerincloud-ccp-vip-HUX-SVC-mgmt"
DNS.15 = "deployerincloud-ccp-vip-HZN-WEB-mgmt"
DNS.16 = "deployerincloud-ccp-vip-KEY-API-mgmt"
DNS.17 = "deployerincloud-ccp-vip-KEYMGR-API-mgmt"
DNS.18 = "deployerincloud-ccp-vip-LOG-API-mgmt"
DNS.19 = "deployerincloud-ccp-vip-LOG-SVR-mgmt"
DNS.20 = "deployerincloud-ccp-vip-MON-API-mgmt"
DNS.21 = "deployerincloud-ccp-vip-NEU-SVR-mgmt"
DNS.22 = "deployerincloud-ccp-vip-NOV-API-mgmt"
DNS.23 = "deployerincloud-ccp-vip-NOV-MTD-mgmt"
DNS.24 = "deployerincloud-ccp-vip-OCT-API-mgmt"
DNS.25 = "deployerincloud-ccp-vip-OPS-WEB-mgmt"
DNS.26 = "deployerincloud-ccp-vip-SHP-API-mgmt"
DNS.27 = "deployerincloud-ccp-vip-SWF-PRX-mgmt"
DNS.28 = "deployerincloud-ccp-vip-admin-CEI-API-mgmt"
DNS.29 = "deployerincloud-ccp-vip-admin-CND-API-mgmt"
DNS.30 = "deployerincloud-ccp-vip-admin-DES-API-mgmt"
DNS.31 = "deployerincloud-ccp-vip-admin-FND-MDB-mgmt"
DNS.32 = "deployerincloud-ccp-vip-admin-FRE-API-mgmt"
DNS.33 = "deployerincloud-ccp-vip-admin-GLA-API-mgmt"
DNS.34 = "deployerincloud-ccp-vip-admin-HEA-ACF-mgmt"
DNS.35 = "deployerincloud-ccp-vip-admin-HEA-ACW-mgmt"
DNS.36 = "deployerincloud-ccp-vip-admin-HEA-API-mgmt"
DNS.37 = "deployerincloud-ccp-vip-admin-HUX-SVC-mgmt"
DNS.38 = "deployerincloud-ccp-vip-admin-HZN-WEB-mgmt"
DNS.39 = "deployerincloud-ccp-vip-admin-KEY-API-mgmt"
DNS.40 = "deployerincloud-ccp-vip-admin-KEYMGR-API-mgmt"
DNS.41 = "deployerincloud-ccp-vip-admin-MON-API-mgmt"
DNS.42 = "deployerincloud-ccp-vip-admin-NEU-SVR-mgmt"
DNS.43 = "deployerincloud-ccp-vip-admin-NOV-API-mgmt"
DNS.44 = "deployerincloud-ccp-vip-admin-OPS-WEB-mgmt"
DNS.45 = "deployerincloud-ccp-vip-admin-SHP-API-mgmt"
DNS.46 = "deployerincloud-ccp-vip-admin-SWF-PRX-mgmt"
DNS.47 = "192.168.245.5"
IP.1 = "192.168.245.5"

=============end of certificate request file.

Note

In the case of a public VIP certificate, please add all the FQDNs you want it to support Currently SUSE OpenStack Cloud does not add the hostname for the external-name specified in network_groups.yml to the certificate request file . However, you can add it to the certificate request file manually. Here we assume that myardana.test is your external-name. In that case you would add this line (to the certificate request file that is shown above in Example 41.1, “Certificate request file”):

DNS.48 = "myardana.test"

Note

Any attempt to use IP addresses rather than FQDNs in certificates must use subject alternate name entries that list both the IP address (needed for Google) and DNS with an IP (needed for a Python bug workaround). Failure to create the certificates in this manner will cause future installations of Go-based tools (such as Cloud Foundry, Stackato and other PaaS components) to fail.

Now that you have a CA and a certificate request file, it is time to generate a CSR.

ardana > export EXAMPLE_SERVER_KEY_FILE='example-internal-cert.key'
ardana > export EXAMPLE_SERVER_CSR_FILE='example-internal-cert.csr'
ardana > export EXAMPLE_SERVER_REQ_FILE=example-internal-cert.req
ardana > openssl req -newkey rsa:2048 -nodes -keyout "$EXAMPLE_SERVER_KEY_FILE" \
-out "$EXAMPLE_SERVER_CSR_FILE" -extensions v3_req -config "$EXAMPLE_SERVER_REQ_FILE"

Note that in production you would usually send the generated example-internal-cert.csr file to your IT department. But in this example you are your own CA, so sign and generate a server certificate.

Note

In a production setting you will not perform this step. You will send the CSR created in the previous section to your company CA or a to a valid public CA and have them sign and send you back the certificate.

This section demonstrates how you would use your own self-signed CA that your created earlier to sign and generate a server certificate. A server certificate is essentially a signed public key, the signer being a CA and trusted by a client. When you install this the signing CA's certificate (called CA certificate or trust chain) on the client machine, you are telling the client to trust this CA, and thereby implicitly trusting any server certificates that are signed by this CA, thus creating a trust anchor.

CA configuration file

When the CA signs the certificate, it uses a configuration file that tells it to verify the CSR. Note that in a production scenario the CA takes care of this for you.

Create a file called openssl.cnf and add the following contents to it.

# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#...

# OpenSSL configuration file.
#

# Establish working directory.

dir = .

[ ca ]
default_ca = CA_default

[ CA_default ]
serial = $dir/serial
database = $dir/index.txt
new_certs_dir = $dir/
certificate = $dir/cacert.pem
private_key = $dir/cakey.pem
unique_subject = no
default_crl_days = 365
default_days = 365
default_md = md5
preserve = no
email_in_dn = no
nameopt = default_ca
certopt = default_ca
policy = policy_match
copy_extensions = copy


[ policy_match ]
countryName = optional
stateOrProvinceName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional

[ req ]
default_bits = 1024 # Size of keys
default_keyfile = key.pem # name of generated keys
default_md = md5 # message digest algorithm
string_mask = nombstr # permitted characters
distinguished_name = req_distinguished_name
req_extensions = v3_req
x509_extensions = v3_ca

[ req_distinguished_name ]
# Variable name Prompt string
#---------------------- ----------------------------------
0.organizationName = Organization Name (company)
organizationalUnitName = Organizational Unit Name (department, division)
emailAddress = Email Address
emailAddress_max = 40
localityName = Locality Name (city, district)
stateOrProvinceName = State or Province Name (full name)
countryName = Country Name (2 letter code)
countryName_min = 2
countryName_max = 2
commonName = Common Name (hostname, IP, or your name)
commonName_max = 64

# Default values for the above, for consistency and less typing.
# Variable name Value
#------------------------------ ------------------------------
0.organizationName_default = Exampleco PLC
localityName_default = Anytown
stateOrProvinceName_default = Anycounty
countryName_default = UK
commonName_default = my-CA

[ v3_ca ]
basicConstraints = CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
subjectAltName = @alt_names

[ v3_req ]
basicConstraints = CA:FALSE
subjectKeyIdentifier = hash

[ alt_names ]

######### end of openssl.cnf #########

Sign and create a server certificate

Now you can sign the server certificate with your CA. Copy the commands below to the command line and execute. This will cause the one file, example-internal-cert.crt, to be created:

ardana > export EXAMPLE_SERVER_CERT_FILE='example-internal-cert.crt'
ardana > export EXAMPLE_SERVER_CSR_FILE='example-internal-cert.csr'
ardana > export EXAMPLE_CA_KEY_FILE='example-CA.key'
ardana > export EXAMPLE_CA_CERT_FILE='example-CA.crt'
ardana > touch index.txt
ardana > openssl rand -hex -out serial 6
ardana > openssl ca -batch -notext -md sha256 -in "$EXAMPLE_SERVER_CSR_FILE" \
-cert "$EXAMPLE_CA_CERT_FILE" \
-keyfile "$EXAMPLE_CA_KEY_FILE" \
-out "$EXAMPLE_SERVER_CERT_FILE" \
-config openssl.cnf -extensions v3_req

Finally, concatenate both the server key and certificate in preparation for uploading to the Cloud Lifecycle Manager.

ardana > cat example-internal-cert.key example-internal-cert.crt > example-internal-cert

Note that you have only created the internal-cert in this example. Repeat the above sequence for example-public-cert. Make sure you use the appropriate certificate request generated by the configuration processor.

The following two files created from the example run above will need to be uploaded to the Cloud Lifecycle Manager and copied into config/tls.

Once on the Cloud Lifecycle Manager, execute the following two copy commands to copy to their respective directories. Note if you had created an external cert, you can copy that in a similar manner, specifying its name using the copy command as well.

ardana > cp example-internal-cert ~/openstack/my_cloud/config/tls/certs/
ardana > cp example-CA.crt ~/openstack/my_cloud/config/tls/cacerts/

Continue with the deployment

Next, log into the Cloud Lifecycle Manager node, and save and commit the changes to the local git repository:

ardana > cd ~/openstack/ardana/ansible
ardana > git add -A
ardana > git commit -m "updated certificate and CA"

Next, rerun the config-processor-run playbook, and run ready-deployment.yml:

ardana > cd ~/openstack/ardana/ansible
ardana > ansible-playbook -i hosts/localhost config-processor-run.yml
ardana > ansible-playbook -i hosts/localhost ready-deployment.yml

If you receive any prompts, enter the required information.

Note

For automated installation (for example CI) you can specify the required passwords on the Ansible command line. For example, the command below will disable encryption by the configuration processor:

ardana > ansible-playbook -i hosts/localhost config-processor-run.yml -e encrypt="" -e rekey=""

Run this series of runbooks to complete the deployment:

ardana > cd ~/scratch/ansible/next/ardana/ansible
ardana > ansible-playbook -i hosts/verb_hosts tls-reconfigure.yml
ardana > ansible-playbook -i hosts/verb_hosts FND-CLU-stop.yml
ardana > ansible-playbook -i hosts/verb_hosts FND-CLU-start.yml
ardana > ansible-playbook -i hosts/verb_hosts monasca-stop.yml
ardana > ansible-playbook -i hosts/verb_hosts monasca-start.yml
ardana > ansible-playbook -i hosts/verb_hosts ardana-reconfigure.yml

By default, the cipher suite is set to: HIGH:!aNULL:!eNULL:!DES:!3DES. This setting is recommended in the OpenStack documentation site. You may override this. To do so, open config/haproxy/defaults.yml and edit it. The parameters can be found under the haproxy_globals list.

- "ssl-default-bind-ciphers HIGH:!aNULL:!eNULL:!DES:!3DES"
- "ssl-default-server-ciphers HIGH:!aNULL:!eNULL:!DES:!3DES"

Make the changes as needed. It is best to keep the two options identical.

You can easily determine if an endpoint is behind TLS. To do so, run the following command, which probes a keystone identity service endpoint that is behind TLS:

ardana > echo | openssl s_client -connect 192.168.245.5:5000 | openssl x509 -fingerprint -noout
depth=0 CN = openstack-vip
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = openstack-vip
verify error:num=27:certificate not trusted
verify return:1
depth=0 CN = openstack-vip
verify error:num=21:unable to verify the first certificate
verify return:1
DONE
SHA1 Fingerprint=C6:46:1E:59:C6:11:BF:72:5E:DD:FC:FF:B0:66:A7:A2:CC:32:1C:B8

The next command probes a MariaDB endpoint that is not behind TLS:

ardana > echo | openssl s_client -connect 192.168.245.5:3306 | openssl x509 -fingerprint -noout
140448358213264:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:795:
unable to load certificate
140454148159120:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:703
:Expecting: TRUSTED CERTIFICATE

You can determine if the trust chain is correctly deployed by running the following commands:

ardana > echo | openssl s_client -connect 192.168.245.9:5000 2>/dev/null | grep code
Verify return code: 21 (unable to verify the first certificate)
echo | openssl s_client -connect 192.168.245.9:5000 \
-CAfile /etc/pki/trust/anchors/ca-certificates/openstack_frontend_cacert.crt 2>/dev/null | grep code
Verify return code: 0 (ok)

Here, the first command produces error 21, which is then fixed by providing the CA certificate file. This verifies that the CA certificate matches the server certificate.

You should leave TLS enabled in production. However, if you need to disable it for any reason, you must change "tls-components" to "components" in network_groups.yml (as shown earlier) and comment out the cert-file. Additionally, if you have a network_groups.yml file from a previous installation, you will not have TLS enabled unless you change "components" to "tls-components" in that file. By default, horizon is configured with TLS in the input model. Note that you should not disable TLS in the input model for horizon as that is a public endpoint and is required. Additionally, you should keep all services behind TLS, but using the input model file network_groups.yml you may turn TLS off for a service for troubleshooting or debugging. TLS should always be enabled for production environments.

If you are using an example input model on a clean install, all supported TLS services will be enabled before deployment of your cloud. If you want to change this setting later, for example, when upgrading, you can change the input model and reconfigure the system. The process is as follows:

Edit the input model network_groups.yml file appropriately, as described above. Then, commit the changes to the git repository:

ardana > cd ~/openstack/ardana/ansible/
ardana > git add -A
ardana > git commit -m "TLS change"

Change directories again and run the configuration processor and ready deployment playbooks:

ardana > ansible-playbook -i hosts/localhost config-processor-run.yml
ardana > ansible-playbook -i hosts/localhost ready-deployment.yml

Change directories again and run the reconfigure playbook:

ardana > cd ~/scratch/ansible/next/ardana/ansible
ardana > ansible-playbook -i hosts/verb_hosts ardana-reconfigure.yml