192.168.124.0/24
(Admin/BMC) Network Address Allocation192.168.125/24
(Storage) Network Address Allocation192.168.123/24
(Private Network/nova-fixed) Network Address Allocation192.168.126/24
(Public Network nova-floating, public) Network Address Allocation192.168.130/24
(Software Defined Network) Network Address AllocationCopyright © 2006– 2022 SUSE LLC and contributors. All rights reserved.
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License :
For SUSE trademarks, see http://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.
All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof.
SUSE® OpenStack Cloud Crowbar is an open source software solution that provides the fundamental capabilities to deploy and manage a cloud infrastructure based on SUSE Linux Enterprise. SUSE OpenStack Cloud Crowbar is powered by OpenStack, the leading community-driven, open source cloud infrastructure project. It seamlessly manages and provisions workloads across a heterogeneous cloud environment in a secure, compliant, and fully-supported manner. The product tightly integrates with other SUSE technologies and with the SUSE maintenance and support infrastructure.
In SUSE OpenStack Cloud Crowbar, there are several different high-level user roles:
Installs and deploys SUSE OpenStack Cloud Crowbar on bare-metal, then installs the operating system and the OpenStack components. For detailed information about the operator's tasks and how to solve them, refer to SUSE OpenStack Cloud Deploying With Crowbar.
Manages projects, users, images, flavors, and quotas within SUSE OpenStack Cloud Crowbar. For detailed information about the administrator's tasks and how to solve them, refer to the OpenStack Administrator Guide and the SUSE OpenStack Cloud Supplement to Administrator Guide and End User Guide.
End user who launches and manages instances, creates snapshots, and uses volumes for persistent storage within SUSE OpenStack Cloud Crowbar. For detailed information about the user's tasks and how to solve them, refer to OpenStack End User Guide and the SUSE OpenStack Cloud Supplement to Administrator Guide and End User Guide.
This guide provides cloud operators with the information needed to deploy and maintain SUSE OpenStack Cloud administrative units, the Administration Server, the Control Nodes, and the Compute and Storage Nodes. The Administration Server provides all services needed to manage and deploy all other nodes in the cloud. The Control Node hosts all OpenStack components needed to operate virtual machines deployed on the Compute Nodes in the SUSE OpenStack Cloud. Each virtual machine (instance) started in the cloud will be hosted on one of the Compute Nodes. Object storage is managed by the Storage Nodes.
Many chapters in this manual contain links to additional documentation resources. These include additional documentation that is available on the system, and documentation available on the Internet.
For an overview of the documentation available for your product and the latest documentation updates, refer to https://documentation.suse.com.
Documentation for our products is available at http://documentation.suse.com, where you can also find the latest updates, and browse or download the documentation in various formats.
In addition, the product documentation
is usually available in your installed system under
/usr/share/doc/manual
. You can also access the
product-specific manuals and the upstream documentation from
the links in the graphical Web interfaces.
The following documentation is available for this product:
Gives an introduction to the SUSE® OpenStack Cloud Crowbar architecture, lists the requirements, and describes how to set up, deploy, and maintain the individual components. Also contains information about troubleshooting, support, and a glossary listing the most important terms and concepts for SUSE OpenStack Cloud Crowbar.
Introduces the OpenStack services and their components.
Also guides you through tasks like managing images, roles, instances, flavors,
volumes, shares, quotas, host aggregates, and viewing cloud resources. To
complete these tasks, use either the graphical Web interface (OpenStack Dashboard,
code name Horizon
) or the OpenStack command line clients.
Describes how to manage images, instances, networks, object containers,
volumes, shares, stacks, and databases. To complete these tasks, use either the graphical Web interface (OpenStack Dashboard, code name
Horizon
) or the OpenStack command line clients.
A supplement to the SUSE OpenStack Cloud Crowbar Administrator Guide and SUSE OpenStack Cloud Crowbar End User Guide. It contains additional information for admins and end users that is specific to SUSE OpenStack Cloud Crowbar.
A manual introducing SUSE OpenStack Cloud Crowbar Monitoring. It is written for everybody interested in SUSE OpenStack Cloud Crowbar Monitoring.
A manual for SUSE OpenStack Cloud Crowbar operators describing how to prepare their OpenStack platform for SUSE OpenStack Cloud Crowbar Monitoring. The manual also describes how the operators use SUSE OpenStack Cloud Crowbar Monitoring for monitoring their OpenStack services.
A manual for system operators describing how to operate SUSE OpenStack Cloud Crowbar Monitoring. The manual also describes how the operators can use SUSE OpenStack Cloud Crowbar Monitoring for monitoring their environment.
Several feedback channels are available:
For services and support options available for your product, refer to http://www.suse.com/support/.
We want to hear your comments about and suggestions for this manual and the other documentation included with this product. If you are reading the HTML version of this guide, use the Comments feature at the bottom of each page in the online documentation at http://documentation.suse.com.
If you are reading the single-page HTML version of this guide, you can use the https://bugzilla.suse.com/. A user account is needed for this.
link next to each section to open a bug report at
For feedback on the documentation of this product, you can also send a
mail to doc-team@suse.com
. Make sure to include the
document title, the product version, and the publication date of the
documentation. To report errors or suggest enhancements, provide a
concise description of the problem and refer to the respective section
number and page (or URL).
The following notices and typographical conventions are used in this documentation:
Vital information you must be aware of before proceeding. Warns you about security issues, potential loss of data, damage to hardware, or physical hazards.
Important information you should be aware of before proceeding.
Additional information, for example about differences in software versions.
Helpful information, like a guideline or a piece of practical advice.
tux >
command
Commands that can be run by any user, including the root
user.
root #
command
Commands that must be run with root
privileges. Often you
can also prefix these commands with the sudo
command to
run them.
/etc/passwd
: directory names and file names
PLACEHOLDER: replace PLACEHOLDER with the actual value
PATH
: the environment variable PATH
ls
, --help
: commands, options, and
parameters
user
: users or groups
Alt, Alt–F1: a key to press or a key combination; keys are shown in uppercase as on a keyboard
, › : menu items, buttons
AMD/Intel This paragraph is only relevant for the AMD64/Intel 64 architecture. The arrows mark the beginning and the end of the text block.
IBM Z, POWER
This paragraph is only relevant for the architectures
z Systems
and POWER
. The arrows
mark the beginning and the end of the text block.
Dancing Penguins (Chapter Penguins, ↑Another Manual): This is a reference to a chapter in another manual.
This documentation is written in SUSEDoc, a subset of DocBook 5. The XML source
files were validated by jing
, processed by
xsltproc
, and converted into XSL-FO using a customized
version of Norman Walsh's stylesheets. The final PDF is formatted through
FOP from Apache Software Foundation. The open source tools
and the environment used to build this documentation are provided by the
DocBook Authoring and Publishing Suite (DAPS). The project's home page can
be found at https://github.com/openSUSE/daps.
The XML source code of this documentation can be found at https://github.com/SUSE-Cloud/doc-cloud.
SUSE OpenStack Cloud Crowbar is a managed cloud infrastructure solution that provides a full stack of cloud deployment and management services.
Before deploying SUSE OpenStack Cloud Crowbar, there are some requirements to meet and architectural decisions to make. Read this chapter thoroughly first, as some decisions need to be made before deploying SUSE OpenStack Cloud, and you cannot change them afterward.
SUSE OpenStack Cloud Crowbar is a managed cloud infrastructure solution that provides a full stack of cloud deployment and management services.
SUSE OpenStack Cloud Crowbar 8 provides the following features:
Open source software that is based on the OpenStack Pike release.
Centralized resource tracking providing insight into activities and capacity of the cloud infrastructure for optimized automated deployment of services.
A self-service portal enabling end users to configure and deploy services as necessary, and to track resource consumption (Horizon).
An image repository from which standardized, pre-configured virtual machines are published (Glance).
Automated installation processes via Crowbar using pre-defined scripts for configuring and deploying the Control Node(s) and Compute and Storage Nodes.
Multi-tenant, role-based provisioning and access control for multiple departments and users within your organization.
APIs enabling the integration of third-party software, such as identity management and billing solutions.
Heterogeneous hypervisor support (Xen and KVM).
An optional monitoring as a service solution, that allows to manage, track, and optimize the cloud infrastructure and the services provided to end users (SUSE OpenStack Cloud Monitoring, Monasca).
SUSE OpenStack Cloud Crowbar is based on SUSE Linux Enterprise Server, OpenStack, Crowbar, and Chef. SUSE Linux Enterprise Server is the underlying operating system for all cloud infrastructure machines (also called nodes). The cloud management layer, OpenStack, works as the “Cloud Operating System”. Crowbar and Chef automatically deploy and manage the OpenStack nodes from a central Administration Server.
SUSE OpenStack Cloud Crowbar is deployed to four different types of machines:
one Administration Server for node deployment and management
one or more Control Nodes hosting the cloud management services
several Compute Nodes on which the instances are started
several Monitoring Node for monitoring services and servers.
The Administration Server provides all services needed to manage and deploy all other nodes in the cloud. Most of these services are provided by the Crowbar tool that—together with Chef—automates all the required installation and configuration tasks. Among the services provided by the server are DHCP, DNS, NTP, PXE, and TFTP.
The Administration Server also hosts the software repositories for SUSE Linux Enterprise Server and SUSE OpenStack Cloud Crowbar, which are needed for node deployment. If no other sources for the software repositories are available, it can host the Subscription Management Tool (SMT), providing up-to-date repositories with updates and patches for all nodes.
The Control Node(s) hosts all OpenStack components needed to orchestrate virtual machines deployed on the Compute Nodes in the SUSE OpenStack Cloud. OpenStack on SUSE OpenStack Cloud uses a MariaDB database, which is hosted on the Control Node(s). The following OpenStack components—if deployed—run on the Control Node(s):
PostgreSQL database.
Image (Glance) for managing virtual images.
Identity (Keystone), providing authentication and authorization for all OpenStack components.
Networking (Neutron), providing “networking as a service” between interface devices managed by other OpenStack services.
Block Storage (Cinder), providing block storage.
OpenStack Dashboard (Horizon), providing the Dashboard, a user Web interface for the OpenStack components.
Compute (Nova) management (Nova controller) including API and scheduler.
Message broker (RabbitMQ).
Swift proxy server plus dispersion tools (health monitor) and Swift ring (index of objects, replicas, and devices). Swift provides object storage.
Hawk, a monitor for a pacemaker cluster in a High Availability (HA) setup.
Heat, an orchestration engine.
Designate provides DNS as a Service (DNSaaS)
Ceilometer server and agents. Ceilometer collects CPU and networking data for billing purposes.
Ironic, the OpenStack bare metal service for provisioning physical machines.
SUSE OpenStack Cloud Crowbar requires a three-node cluster for any production deployment since it leverages a MariaDB Galera Cluster for high availability.
We recommend deploying certain parts of Networking (Neutron) on separate nodes for production clouds. See Section 12.10, “Deploying Neutron” for details.
You can separate authentication and authorization services from other cloud services, for stronger security, by hosting Identity (Keystone) on a separate node. Hosting Block Storage (Cinder, particularly the cinder-volume role) on a separate node when using local disks for storage enables you to customize your storage and network hardware to best meet your requirements.
If you plan to move a service from one Control Node to another, we strongly recommended shutting down or saving all instances before doing so. Restart them after having successfully re-deployed the services. Moving services also requires stopping them manually on the original Control Node.
The Compute Nodes are the pool of machines on which your instances are running. These machines need to be equipped with a sufficient number of CPUs and enough RAM to start several instances. They also need to provide sufficient hard disk space, see Section 2.2.2.3, “Compute Nodes” for details. The Control Node distributes instances within the pool of Compute Nodes and provides them with the necessary network resources. The OpenStack component Compute (Nova) runs on the Compute Nodes and provides the means for setting up, starting, and stopping virtual machines.
SUSE OpenStack Cloud Crowbar supports several hypervisors, including KVM, VMware vSphere, and Xen. Each image that is started with an instance is bound to one hypervisor. Each Compute Node can only run one hypervisor at a time. You will choose which hypervisor to run on each Compute Node when deploying the Nova barclamp.
The Storage Nodes are the pool of machines providing object or block storage. Object storage is provided by the OpenStack Swift component, while block storage is provided by Cinder. The latter supports several back-ends, including Ceph, that are deployed during the installation. Deploying Swift and Ceph is optional.
The Monitoring Node is the node that has the monasca-server
role assigned.
It hosts most services needed for SUSE OpenStack Cloud
Monitoring, our Monasca-based monitoring and logging solution. The following
services run on this node:
The Monasca Web API that is used for sending metrics by Monasca agents, and retrieving metrics with the Monasca command line client and the Monasca Grafana dashboard.
A Kafka instance used exclusively by SUSE OpenStack Cloud Monitoring.
Stores metrics and alarms in InfluxDB.
Consumes alarms sent by the Threshold Engine and sends notifications (e.g. via email).
Based on Apache Storm. Computes thresholds on metrics and handles alarming.
A Cassandra database for storing metrics alarm history.
A dedicated MariaDB instance used only for monitoring related data.
The Monasca Web API that is used for sending log entries by Monasca agents, and retrieving log entries with the Kibana Server.
Transforms raw log entries sent to the Log API into a format suitable for storage.
Sends metrics about high severity log messages to the Monitoring API.
Stores logs processed by Monasca Log Transformer in the Log Database.
A graphical web frontend for querying the Log Database.
An Elasticsearch database for storing logs.
Cluster synchronization for Kafka and Storm.
Currently there can only be one Monitoring node. Clustering support is planned for a future release. We strongly recommend using a dedicated physical node without any other services as a Monitoring Node.
A failure of components in SUSE OpenStack Cloud Crowbar can lead to system downtime and data loss. To prevent this, set up a High Availability (HA) cluster consisting of several nodes. You can assign certain roles to this cluster instead of assigning them to individual nodes. As of SUSE OpenStack Cloud Crowbar 8, Control Nodes and Compute Nodes can be made highly available.
For all HA-enabled roles, their respective functions are automatically handled by the clustering software SUSE Linux Enterprise High Availability Extension. The High Availability Extension uses the Pacemaker cluster stack with Pacemaker as cluster resource manager, and Corosync as the messaging/infrastructure layer.
View the cluster status and configuration with the cluster
management tools HA Web Console (Hawk) or the
crm
shell.
Use the cluster management tools only for viewing. All of the clustering configuration is done automatically via Crowbar and Chef. If you change anything via the cluster management tools you risk breaking the cluster. Changes done there may be reverted by the next run of Chef anyway.
A failure of the OpenStack infrastructure services (running on the Control Nodes) can be critical and may cause downtime within the cloud. For more information on making those services highly-available and avoiding other potential points of failure in your cloud setup, refer to Section 2.6, “High Availability”.
Before deploying SUSE OpenStack Cloud Crowbar, there are some requirements to meet and architectural decisions to make. Read this chapter thoroughly first, as some decisions need to be made before deploying SUSE OpenStack Cloud, and you cannot change them afterward.
SUSE OpenStack Cloud Crowbar requires a complex network setup consisting of several networks that are configured during installation. These networks are for exclusive cloud usage. You need a router to access them from an existing network.
The network configuration on the nodes in the SUSE OpenStack Cloud network is entirely controlled by Crowbar. Any network configuration not created with Crowbar (for example, with YaST) will automatically be overwritten. After the cloud is deployed, network settings cannot be changed.
The following networks are pre-defined when setting up SUSE OpenStack Cloud Crowbar. The IP addresses listed are the default addresses and can be changed using the YaST Crowbar module (see Chapter 7, Crowbar Setup). It is also possible to customize the network setup by manually editing the network barclamp template. See Section 7.5, “Custom Network Configuration” for detailed instructions.
A private network to access the Administration Server and all nodes for administration purposes. The default setup also allows access to the BMC (Baseboard Management Controller) data via IPMI (Intelligent Platform Management Interface) from this network. If required, BMC access can be swapped to a separate network.
You have the following options for controlling access to this network:
Do not allow access from the outside and keep the admin network completely separated.
Allow access to the Administration Server from a single network (for example, your company's administration network) via the “bastion network” option configured on an additional network card with a fixed IP address.
Allow access from one or more networks via a gateway.
Private SUSE OpenStack Cloud internal virtual network. This network is used by Ceph and Swift only. It should not be accessed by users.
Private SUSE OpenStack Cloud internal virtual network. This network is used for inter-instance communication and provides access to the outside world for the instances. The required gateway is automatically provided by SUSE OpenStack Cloud.
The only public network provided by SUSE OpenStack Cloud. You can access the Nova Dashboard and all instances (provided they have been equipped with floating IP addresses) on this network. This network can only be accessed via a gateway, which must be provided externally. All SUSE OpenStack Cloud users and administrators must have access to the public network.
Private SUSE OpenStack Cloud internal virtual network. This network is used
when Neutron is configured to use openvswitch with
GRE
tunneling for the virtual networks. It should
not be accessible to users.
The Monasca monitoring node needs to have an interface on both the
admin network and the public network. Monasca's backend services will
listen on the admin network, the API services
(openstack-monasca-api
,
openstack-monasca-log-api
) will listen on all
interfaces. openstack-monasca-agent
and
openstack-monasca-log-agent
will send their logs
and metrics to the
monasca-api
/monasca-log-api
services to the monitoring node's public network IP address.
For security reasons, protect the following networks from external access:
Especially traffic from the cloud instances must not be able to pass through these networks.
As of SUSE OpenStack Cloud Crowbar 8, using a VLAN for the admin network is only supported on a native/untagged VLAN. If you need VLAN support for the admin network, it must be handled at switch level.
When changing the network configuration with YaST or by editing
/etc/crowbar/network.json
you can define VLAN
settings for each network. For the networks nova-fixed
and nova-floating
, however, special rules apply:
nova-fixed: The setting will be ignored. However, VLANs will automatically be used if deploying Neutron with VLAN support (using the plugins linuxbridge, openvswitch plus VLAN, or cisco plus VLAN). In this case, you need to specify a correct for this network.
nova-floating: When using a VLAN for
nova-floating
(which is the default), the and settings for
and must be
the same. When not using a VLAN for nova-floating
, it
must have a different physical network interface than the
nova_fixed
network.
As of SUSE OpenStack Cloud Crowbar 8, IPv6 is not supported. This applies to the cloud internal networks and to the instances. You must use static IPv4 addresses for all network interfaces on the Admin Node, and disable IPv6 before deploying Crowbar on the Admin Node.
The following diagram shows the pre-defined SUSE OpenStack Cloud network in more detail. It demonstrates how the OpenStack nodes and services use the different networks.
The default networks set up in SUSE OpenStack Cloud are class C networks with 256 IP addresses each. This limits the maximum number of instances that can be started simultaneously. Addresses within the networks are allocated as outlined in the following table. Use the YaST Crowbar module to make customizations (see Chapter 7, Crowbar Setup). The last address in the IP address range of each network is always reserved as the broadcast address. This assignment cannot be changed.
For an overview of the minimum number of IP addresses needed for each of the ranges in the network settings, see Table 2.1, “Minimum Number of IP Addresses for Network Ranges”.
Network |
Required Number of IP addresses |
---|---|
Admin Network |
|
Public Network |
|
BMC Network |
|
Software Defined Network |
|
The default network proposal as described below limits the maximum number of Compute Nodes to 80, the maximum number of floating IP addresses to 61 and the maximum number of addresses in the nova_fixed network to 204.
To overcome these limitations you need to reconfigure the network setup by using appropriate address ranges. Do this by either using the YaST Crowbar module as described in Chapter 7, Crowbar Setup, or by manually editing the network template file as described in Section 7.5, “Custom Network Configuration”.
192.168.124.0/24
(Admin/BMC) Network Address Allocation #
Function |
Address |
Remark |
---|---|---|
router |
|
Provided externally. |
admin |
|
Fixed addresses reserved for the Administration Server. |
DHCP |
|
Address range reserved for node allocation/installation. Determines the maximum number of parallel allocations/installations. |
host |
|
Fixed addresses for the OpenStack nodes. Determines the maximum number of OpenStack nodes that can be deployed. |
bmc vlan host |
|
Fixed address for the BMC VLAN. Used to generate a VLAN tagged interface on the Administration Server that can access the BMC network. The BMC VLAN must be in the same ranges as BMC, and BMC must have VLAN enabled. |
bmc host |
|
Fixed addresses for the OpenStack nodes. Determines the maximum number of OpenStack nodes that can be deployed. |
switch |
|
This range is not used in current releases and might be removed in the future. |
192.168.125/24
(Storage) Network Address Allocation #
Function |
Address |
Remark |
---|---|---|
host |
|
Each Storage Node will get an address from this range. |
192.168.123/24
(Private Network/nova-fixed) Network Address Allocation #
Function |
Address |
Remark |
---|---|---|
DHCP |
|
Address range for instances, routers and DHCP/DNS agents. |
192.168.126/24
(Public Network nova-floating, public) Network Address Allocation #
Function |
Address |
Remark |
---|---|---|
router |
|
Provided externally. |
public host |
|
Public address range for external SUSE OpenStack Cloud components such as the OpenStack Dashboard or the API. |
floating host |
|
Floating IP address range. Floating IP addresses can be manually assigned to a running instance to allow to access the guest from the outside. Determines the maximum number of instances that can concurrently be accessed from the outside. The nova_floating network is set up with a netmask of 255.255.255.192, allowing a maximum number of 61 IP addresses. This range is pre-allocated by default and managed by Neutron. |
192.168.130/24
(Software Defined Network) Network Address Allocation #
Function |
Address |
Remark |
---|---|---|
host |
|
If Neutron is configured with |
Addresses not used in the ranges mentioned above can be used to add additional servers with static addresses to SUSE OpenStack Cloud. Such servers can be used to provide additional services. A SUSE Manager server inside SUSE OpenStack Cloud, for example, must be configured using one of these addresses.
SUSE OpenStack Cloud Crowbar supports different network modes defined in Crowbar: single
, dual
, and
team
. As of SUSE OpenStack Cloud Crowbar 8, the networking mode
is applied to all nodes and the Administration Server. That means that all
machines need to meet the hardware requirements for the chosen mode. The
network mode can be configured using the YaST Crowbar module
(Chapter 7, Crowbar Setup). The network mode cannot
be changed after the cloud is deployed.
Other, more flexible network mode setups can be configured by manually editing the Crowbar network configuration files. See Section 7.5, “Custom Network Configuration” for more information. SUSE or a partner can assist you in creating a custom setup within the scope of a consulting services agreement (see http://www.suse.com/consulting/ for more information on SUSE consulting).
Network device bonding is required for an HA setup of SUSE OpenStack Cloud Crowbar. If you are planning to move your cloud to an HA setup at a later point in time, make sure to use a network mode in the YaST Crowbar that supports network device bonding.
Otherwise a migration to an HA setup is not supported.
Dual mode needs two Ethernet cards (on all nodes but Administration Server) to completely separate traffic between the Admin Network and the public network:
Team mode is similar to single mode, except that you combine several Ethernet cards to a “bond” (network device bonding). Team mode needs two or more Ethernet cards.
When using team mode, you must choose a “bonding policy” that defines how to use the combined Ethernet cards. You can either set them up for fault tolerance, performance (load balancing), or a combination of both.
Enabling access to the Administration Server from another network requires an external gateway. This option offers maximum flexibility, but requires additional hardware and may be less secure than you require. Therefore SUSE OpenStack Cloud offers a second option for accessing the Administration Server: the bastion network. You only need a dedicated Ethernet card and a static IP address from the external network to set it up.
The bastion network setup (see Section 7.3.1, “Setting Up a Bastion Network” for setup instructions) enables logging in to the Administration Server via SSH from the company network. A direct login to other nodes in the cloud is not possible. However, the Administration Server can act as a “jump host”: First log in to the Administration Server via SSH, then log in via SSH to other nodes.
The Administration Server acts as a name server for all nodes in the cloud. If the Administration Server has access to the outside, then you can add additional name servers that are automatically used to forward requests. If additional name servers are found on your cloud deployment, the name server on the Administration Server is automatically configured to forward requests for non-local records to these servers.
The Administration Server must have a fully qualified host name. The domain name you specify is used for the DNS zone. It is required to use a sub-domain such as cloud.example.com. The Administration Server must have authority over the domain it is on so that it can create records for discovered nodes. As a result, it will not forward requests for names it cannot resolve in this domain, and thus cannot resolve names for the second-level domain, .e.g. example.com, other than for nodes in the cloud.
This host name must not be changed after SUSE OpenStack Cloud has been deployed. The OpenStack nodes are named after their MAC address by default, but you can provide aliases, which are easier to remember when allocating the nodes. The aliases for the OpenStack nodes can be changed at any time. It is useful to have a list of MAC addresses and the intended use of the corresponding host at hand when deploying the OpenStack nodes.
When talking about “persistent storage” on SUSE OpenStack Cloud Crowbar, there are two completely different aspects to discuss: 1) the block and object storage services SUSE OpenStack Cloud offers, 2) the hardware related storage aspects on the different node types.
Block and object storage are persistent storage models where files or images are stored until they are explicitly deleted. SUSE OpenStack Cloud also offers ephemeral storage for images attached to instances. These ephemeral images only exist during the life of an instance and are deleted when the guest is terminated. See Section 2.2.2.3, “Compute Nodes” for more information.
SUSE OpenStack Cloud offers two different types of services for persistent storage: object and block storage. Object storage lets you upload and download files (similar to an FTP server), whereas a block storage provides mountable devices (similar to a hard disk partition). SUSE OpenStack Cloud provides a repository to store the virtual disk images used to start instances.
The OpenStack object storage service is called Swift. The storage component of Swift (swift-storage) must be deployed on dedicated nodes where no other cloud services run. Deploy at least two Swift nodes to provide redundant storage. SUSE OpenStack Cloud Crowbar is configured to always use all unused disks on a node for storage.
Swift can optionally be used by Glance, the service that manages the images used to boot the instances. Offering object storage with Swift is optional.
Block storage on SUSE OpenStack Cloud is provided by Cinder. Cinder can use a variety of storage back-ends, including network storage solutions like NetApp or EMC. It is also possible to use local disks for block storage. A list of drivers available for Cinder and the features supported for each driver is available from the CinderSupportMatrix at https://wiki.openstack.org/wiki/CinderSupportMatrix. SUSE OpenStack Cloud Crowbar 8 ships with OpenStack Pike.
Alternatively, Cinder can use Ceph RBD as a back-end. Ceph offers data security and speed by storing the the content on a dedicated Ceph cluster.
Glance provides a catalog and repository for virtual disk images used to start the instances. Glance is installed on a Control Node. It uses Swift, Ceph, or a directory on the Control Node to store the images. The image directory can either be a local directory or an NFS share.
Each node in SUSE OpenStack Cloud needs sufficient disk space to store both the operating system and additional data. Requirements and recommendations for the various node types are listed below.
The operating system will always be installed on the first hard disk. This is the disk that is listed first in the BIOS, the one from which the machine will boot. Make sure that the hard disk the operating system is installed on will be recognized as the first disk.
If you store the update repositories directly on the Administration Server (see
Section 2.5.2, “Product and Update Repositories”), we recommend mounting /srv
on a separate partition or volume with a minimum of 30 GB space.
Log files from all nodes in SUSE OpenStack Cloud are stored on the Administration Server
under /var/log
(see
Section 19.1, “On the Administration Server” for a complete list).
The message service RabbitMQ requires 1 GB of free space
in /var
.
Depending on how the services are set up, Glance and Cinder may require additional disk space on the Control Node on which they are running. Glance may be configured to use a local directory, whereas Cinder may use a local image file for storage. For performance and scalability reasons this is only recommended for test setups. Make sure there is sufficient free disk space available if you use a local file for storage.
Cinder may be configured to use local disks for storage (configuration option raw
). If you choose this setup, we recommend deploying the role to one or more dedicated Control Nodes. Those should be equipped with several disks providing sufficient storage space. It may also be necessary to equip this node with two or more bonded network cards, since it will generate heavy network traffic. Bonded network cards require a special setup for this node. For details, refer to Section 7.5, “Custom Network Configuration”.
Live migration for Xen instances requires exporting /var/lib/nova/instances
on the Control Node hosting nova-controller
. This directory will host a copy of the root disk of all Xen instances in the cloud and needs to have sufficient disk space. We strongly recommended using a separate block device for this directory, preferably a RAID device to ensure data security.
Unless an instance is started via “Boot from Volume”, it is started with at least one disk, which is a copy of the image from which it has been started. Depending on the flavor you start, the instance may also have a second, so-called “ephemeral” disk. The size of the root disk depends on the image itself. Ephemeral disks are always created as sparse image files that grow up to a defined size as they are “filled”. By default ephemeral disks have a size of 10 GB.
Both disks, root images and ephemeral disk, are directly bound to the
instance and are deleted when the instance is terminated.
These disks are bound to the Compute Node on which the
instance has been started. The disks are created under
/var/lib/nova
on the Compute Node. Your
Compute Nodes should be equipped with enough disk space to store the
root images and ephemeral disks.
Do not confuse ephemeral disks with persistent block storage. In addition to an ephemeral disk, which is automatically provided with most instance flavors, you can optionally add a persistent storage device provided by Cinder. Ephemeral disks are deleted when the instance terminates, while persistent storage devices can be reused in another instance.
The maximum disk space required on a compute node depends on the available flavors. A flavor specifies the number of CPUs, RAM, and disk size of an instance. Several flavors ranging from
(1 CPU, 512 MB RAM, no ephemeral disk) to (8 CPUs, 8 GB RAM, 10 GB ephemeral disk) are available by default. Adding custom flavors, and editing and deleting existing flavors is also supported.To calculate the minimum disk space needed on a compute node, you need to determine the highest disk-space-to-RAM ratio from your flavors. For example:
Flavor small: 2 GB RAM, 100 GB ephemeral disk => 50 GB disk /1 GB RAM |
Flavor large: 8 GB RAM, 200 GB ephemeral disk => 25 GB disk /1 GB RAM |
So, 50 GB disk /1 GB RAM is the ratio that matters. If you multiply that value by the amount of RAM in GB available on your compute node, you have the minimum disk space required by ephemeral disks. Pad that value with sufficient space for the root disks plus a buffer to leave room for flavors with a higher disk-space-to-RAM ratio in the future.
The scheduler that decides in which node an instance is started does not check for available disk space. If there is no disk space left on a compute node, this will not only cause data loss on the instances, but the compute node itself will also stop operating. Therefore you must make sure all compute nodes are equipped with enough hard disk space.
The block storage service Ceph RBD and the object storage service Swift need to be deployed onto dedicated nodes—it is not possible to mix these services. The Swift component requires at least two machines (more are recommended) to store data redundantly. For information on hardware requirements for Ceph, see https://documentation.suse.com/ses/5.5/single-html/ses-deployment/#storage-bp-hwreq
Each Ceph/Swift Storage Node needs at least two hard disks. The first one will be used for the operating system installation, while the others can be used for storage. We recommend equipping the storage nodes with as many disks as possible.
Using RAID on Swift storage nodes is not supported. Swift takes care of redundancy and replication on its own. Using RAID with Swift would also result in a huge performance penalty.
Whenever non-public data travels over a network it must be encrypted. Encryption protects the integrity and confidentiality of data. Therefore you should enable SSL support when deploying SUSE OpenStack Cloud to production. (SSL is not enabled by default as it requires you to provide certificates.) The following services (and their APIs, if available) can use SSL:
Cinder
Horizon
Glance
Heat
Keystone
Manila
Neutron
Nova
Swift
VNC
RabbitMQ
Ironic
Magnum
You have two options for deploying your SSL certificates. You may use a single shared certificate for all services on each node, or provide individual certificates for each service. The minimum requirement is a single certificate for the Control Node and all services installed on it.
Certificates must be signed by a trusted authority. Refer to https://documentation.suse.com/sles/12-SP5/single-html/SLES-admin/#sec-apache2-ssl for instructions on how to create and sign them.
Each SSL certificate is issued for a certain host name and, optionally,
for alternative host names (via the AlternativeName
option). Each publicly available node in SUSE OpenStack Cloud has two host
names—an internal and a public one. The SSL certificate needs
to be issued for both internal and public names.
The internal name has the following scheme:
dMACADDRESS.FQDN
MACADDRESS is the MAC address of the
interface used to boot the machine via PXE. All letters are turned
lowercase and all colons are replaced with dashes. For example,
00-00-5E-00-53-00
. FQDN is
the fully qualified domain name. An example name looks like this:
d00-00-5E-00-53-00.example.com
Unless you have entered a custom Section 11.2, “Node Installation” for details),
the public name is the same as the internal name prefixed by
public
:
public-d00-00-5E-00-53-00.example.com
To look up the node names open the Crowbar Web interface and click the name of a node in the
. The names are listed as and .Precise hardware requirements can only be listed for the Administration Server and the OpenStack Control Node. The requirements of the OpenStack Compute and Storage Nodes depends on the number of concurrent instances and their virtual hardware equipment.
A minimum of three machines are required for a SUSE OpenStack Cloud: one Administration Server, one Control Node, and one Compute Node. You also need a gateway providing access to the public network. Deploying storage requires additional nodes: at least two nodes for Swift and a minimum of four nodes for Ceph.
Deploying SUSE OpenStack Cloud functions to virtual machines is only supported for the Administration Server—all other nodes need to be physical hardware. Although the Control Node can be virtualized in test environments, this is not supported for production systems.
SUSE OpenStack Cloud currently only runs on x86_64
hardware.
Architecture: x86_64.
RAM: at least 4 GB, 8 GB recommended. The demand for memory depends on the total number of nodes in SUSE OpenStack Cloud—the higher the number of nodes, the more RAM is needed. A deployment with 50 nodes requires a minimum of 24 GB RAM for each Control Node.
Hard disk: at least 50 GB. We recommend putting
/srv
on a separate partition with at least
additional 30 GB of space. Alternatively, you can mount the update
repositories from another server (see Section 2.5.2, “Product and Update Repositories” for details).
Number of network cards: 1 for single and dual mode, 2 or more for team mode. Additional networks such as the bastion network and/or a separate BMC network each need an additional network card. See Section 2.1, “Network” for details.
Can be deployed on physical hardware or a virtual machine.
Architecture: x86_64.
RAM: at least 8 GB, 12 GB when deploying a single Control Node, and 32 GB recommended.
Number of network cards: 1 for single mode, 2 for dual mode, 2 or more for team mode. See Section 2.1, “Network” for details.
Hard disk: See Section 2.2.2.2, “Control Nodes”.
The Compute Nodes need to be equipped with a sufficient amount of RAM and CPUs, matching the numbers required by the maximum number of instances running concurrently. An instance started in SUSE OpenStack Cloud cannot share resources from several physical nodes. It uses the resources of the node on which it was started. So if you offer a flavor (see Flavor for a definition) with 8 CPUs and 12 GB RAM, at least one of your nodes should be able to provide these resources. Add 1 GB RAM for every two nodes (including Control Nodes and Storage Nodes) deployed in your cloud.
See Section 2.2.2.3, “Compute Nodes” for storage requirements.
Usually a single CPU and a minimum of 4 GB RAM are sufficient for the Storage Nodes. Memory requirements increase depending on the total number of nodes in SUSE OpenStack Cloud—the higher the number of nodes, the more RAM you need. A deployment with 50 nodes requires a minimum of 20 GB for each Storage Node. If you use Ceph as storage, the storage nodes should be equipped with an additional 2 GB RAM per OSD (Ceph object storage daemon).
For storage requirements, see Section 2.2.2.4, “Storage Nodes (optional)”.
The Monasca Node is a dedicated physical machine that runs the
monasca-server
role. This node is used for
SUSE OpenStack Cloud Crowbar Monitoring. Hardware requirements for the Monasca Node are as
follows:
Architecture: x86_64
RAM: At least 32 GB, 64 GB or more is recommended
CPU: At least 8 cores, 16 cores or more is recommended
Hard Disk: SSD is strongly recommended
The following formula can be used to calculate the required disk space:
200 GB + ["number of nodes" * "retention period" * ("space for log data/day" + "space for metrics data/day") ]
The recommended values for the formula are as follows:
Retention period = 60 days for InfluxDB and Elasticsearch
Space for daily log data = 2GB
Space for daily metrics data = 50MB
The formula is based on the following log data assumptions:
Approximately 50 log files per node
Approximately 1 log entry per file per sec
200 bytes in size
The formula is based on the following metrics data assumptions:
400 metrics per node
Time interval of 30 seconds
20 bytes in size
The formula provides only a rough estimation of the required disk space. There are several factors that can affect disk space requirements. This includes the exact combination of services that run on your OpenStack node actual cloud usage pattern, and whether any or all services have debug logging enabled.
All nodes and the Administration Server in SUSE OpenStack Cloud run on SUSE Linux Enterprise Server 12 SP3. Subscriptions for the following components are available as one- or three-year subscriptions including priority support:
SUSE OpenStack Cloud Crowbar Control Node + SUSE OpenStack Cloud Crowbar Administration Server (including entitlements for High Availability and SUSE Linux Enterprise Server 12 SP3)
Additional SUSE OpenStack Cloud Crowbar Control Node (including entitlements for High Availability and SUSE Linux Enterprise Server 12 SP3)
SUSE OpenStack Cloud Crowbar Compute Node (excluding entitlements for High Availability and SUSE Linux Enterprise Server 12 SP3)
SUSE OpenStack Cloud Crowbar Swift node (excluding entitlements for High Availability and SUSE Linux Enterprise Server 12 SP3)
SUSE Linux Enterprise Server 12 SP3, HA entitlements for Compute Nodes and Swift Storage Nodes, and entitlements for guest operating systems need to be purchased separately. Refer to http://www.suse.com/products/suse-openstack-cloud/how-to-buy/ for more information on licensing and pricing.
Running an external Ceph cluster (optional) with SUSE OpenStack Cloud requires an additional SUSE Enterprise Storage subscription. Refer to https://www.suse.com/products/suse-enterprise-storage/ and https://www.suse.com/products/suse-openstack-cloud/frequently-asked-questions for more information.
A SUSE account is needed for product registration and access to update repositories. If you do not already have one, go to http://www.suse.com/ to create it.
SUSE OpenStack Cloud Crowbar can be extended by SUSE Enterprise Storage for setting up a Ceph cluster providing block storage services. To store virtual disks for instances, SUSE OpenStack Cloud uses block storage provided by the Cinder module. Cinder itself needs a back-end providing storage. In production environments this usually is a network storage solution. Cinder can use a variety of network storage back-ends, among them solutions from EMC, Fujitsu, or NetApp. In case your organization does not provide a network storage solution that can be used with SUSE OpenStack Cloud, you can set up a Ceph cluster with SUSE Enterprise Storage. SUSE Enterprise Storage provides a reliable and fast distributed storage architecture using commodity hardware platforms.
You need seven software repositories to deploy SUSE OpenStack Cloud and to keep a running SUSE OpenStack Cloud up-to-date. This includes the static product repositories, which do not change over the product life cycle, and the update repositories, which constantly change. The following repositories are needed:
The SUSE Linux Enterprise Server 12 SP3 product repository is a copy of the installation media (DVD #1) for SUSE Linux Enterprise Server. As of SUSE OpenStack Cloud Crowbar 8 it is required to have it available locally on the Administration Server. This repository requires approximately 3.5 GB of hard disk space.
The SUSE OpenStack Cloud Crowbar 8 product repository is a copy of the installation media (DVD #1) for SUSE OpenStack Cloud. It can either be made available remotely via HTTP, or locally on the Administration Server. We recommend the latter since it makes the setup of the Administration Server easier. This repository requires approximately 500 MB of hard disk space.
This repository is created automatically on the Administration Server when you install the SUSE OpenStack Cloud add-on product. It serves as a repository for “Program Temporary Fixes” (PTF), which are part of the SUSE support program.
The SUSE Linux Enterprise Server and SUSE OpenStack Cloud Crowbar repositories contain all binary RPMs from the installation media, plus pattern information and support status metadata. These repositories are served from SUSE Customer Center and need to be kept in synchronization with their sources. Make them available remotely via an existing SMT or SUSE Manager server. Alternatively, make them available locally on the Administration Server by installing a local SMT server, by mounting or synchronizing a remote directory, or by copying them.
These repositories contain maintenance updates to packages in the corresponding Pool repositories. These repositories are served from SUSE Customer Center and need to be kept synchronized with their sources. Make them available remotely via an existing SMT or SUSE Manager server, or locally on the Administration Server by installing a local SMT server, by mounting or synchronizing a remote directory, or by regularly copying them.
As explained in Section 2.6, “High Availability”, Control Nodes in SUSE OpenStack Cloud can optionally be made highly available with the SUSE Linux Enterprise High Availability Extension. The following repositories are required to deploy SLES High Availability Extension nodes:
The pool repositories contain all binary RPMs from the installation media, plus pattern information and support status metadata. These repositories are served from SUSE Customer Center and need to be kept in synchronization with their sources. Make them available remotely via an existing SMT or SUSE Manager server. Alternatively, make them available locally on the Administration Server by installing a local SMT server, by mounting or synchronizing a remote directory, or by copying them.
These repositories contain maintenance updates to packages in the corresponding pool repositories. These repositories are served from SUSE Customer Center and need to be kept synchronized with their sources. Make them available remotely via an existing SMT or SUSE Manager server, or locally on the Administration Server by installing a local SMT server, by mounting or synchronizing a remote directory, or by regularly copying them.
The product repositories for SUSE Linux Enterprise Server 12 SP3 and SUSE OpenStack Cloud Crowbar 8 do not change during the life cycle of a product. Thus, they can be copied to the destination directory from the installation media. However, the pool and update repositories must be kept synchronized with their sources on the SUSE Customer Center. SUSE offers two products that synchronize repositories and make them available within your organization: SUSE Manager (http://www.suse.com/products/suse-manager/, and Subscription Management Tool (which ships with SUSE Linux Enterprise Server 12 SP3).
All repositories must be served via HTTP to be available for SUSE OpenStack Cloud deployment. Repositories that are installed on the Administration Server are made available by the Apache Web server running on the Administration Server. If your organization already uses SUSE Manager or SMT, you can use the repositories provided by these servers.
Making the repositories locally available on the Administration Server has the advantage of a simple network setup within SUSE OpenStack Cloud, and it allows you to seal off the SUSE OpenStack Cloud network from other networks in your organization. Hosting the repositories on a remote server has the advantage of using existing resources and services, and it makes setting up the Administration Server much easier. However, this requires a custom network setup for SUSE OpenStack Cloud, since the Administration Server needs access to the remote server.
The SMT server, shipping with SUSE Linux Enterprise Server 12 SP3, regularly synchronizes repository data from SUSE Customer Center with your local host. Installing the SMT server on the Administration Server is recommended if you do not have access to update repositories from elsewhere within your organization. This option requires the Administration Server to have Internet access.
If you already run an SMT server within your organization, you can use it within SUSE OpenStack Cloud. When using a remote SMT server, update repositories are served directly from the SMT server. Each node is configured with these repositories upon its initial setup.
The SMT server needs to be accessible from the Administration Server and all nodes in SUSE OpenStack Cloud (via one or more gateways). Resolving the server's host name also needs to work.
Each client that is managed by SUSE Manager needs to register with the SUSE Manager server. Therefore the SUSE Manager support can only be installed after the nodes have been deployed. SUSE Linux Enterprise Server 12 SP3 must be set up for autoinstallation on the SUSE Manager server in order to use repositories provided by SUSE Manager during node deployment.
The server needs to be accessible from the Administration Server and all nodes in SUSE OpenStack Cloud (via one or more gateways). Resolving the server's host name also needs to work.
If you can access existing repositories from within your company network from the Administration Server, you have the following options: mount, synchronize, or manually transfer these repositories to the required locations on the Administration Server.
Several components and services in SUSE OpenStack Cloud Crowbar are potentially single points of failure that may cause system downtime and data loss if they fail.
SUSE OpenStack Cloud Crowbar provides various mechanisms to ensure that the crucial components and services are highly available. The following sections provide an overview of components on each node that can be made highly available. For making the Control Node functions and the Compute Nodes highly available, SUSE OpenStack Cloud Crowbar uses the cluster software SUSE Linux Enterprise High Availability Extension. Make sure to thoroughly read Section 2.6.5, “Cluster Requirements and Recommendations” to learn about additional requirements for high availability deployments.
The Administration Server provides all services needed to manage and deploy all other nodes in the cloud. If the Administration Server is not available, new cloud nodes cannot be allocated, and you cannot add new roles to cloud nodes.
However, only two services on the Administration Server are single points of failure, without which the cloud cannot continue to run properly: DNS and NTP.
To avoid DNS and NTP as potential points of failure, deploy the roles
dns-server
and
ntp-server
to multiple nodes.
If any configured DNS forwarder or NTP external server is not
reachable through the admin network from these nodes, allocate an
address in the public network for each node that has the
dns-server
and
ntp-server
roles:
crowbar network allocate_ip default `hostname -f` public host
Then the nodes can use the public gateway to reach the external
servers. The change will only become effective after the next run of
chef-client
on the affected nodes.
The Control Node(s) usually run a variety of services without which the cloud would not be able to run properly.
To prevent the cloud from avoidable downtime if one or more Control Nodes fail, you can make the following roles highly available:
database-server
(database
barclamp)
keystone-server
(keystone
barclamp)
rabbitmq-server
(rabbitmq
barclamp)
swift-proxy
(swift
barclamp)
glance-server
(glance
barclamp)
cinder-controller
(cinder
barclamp)
neutron-server
(neutron
barclamp)
neutron-network
(neutron
barclamp)
nova-controller
(nova
barclamp)
nova_dashboard-server
(nova_dashboard
barclamp)
ceilometer-server
(ceilometer
barclamp)
ceilometer-polling
(ceilometer
barclamp)
heat-server
(heat
barclamp)
Instead of assigning these roles to individual cloud nodes, you can assign them to one or several High Availability clusters. SUSE OpenStack Cloud Crowbar will then use the Pacemaker cluster stack (shipped with the SUSE Linux Enterprise High Availability Extension) to manage the services. If one Control Node fails, the services will fail over to another Control Node. For details on the Pacemaker cluster stack and the SUSE Linux Enterprise High Availability Extension, refer to the High Availability Guide, available at https://documentation.suse.com/sle-ha/12-SP5/. Note that SUSE Linux Enterprise High Availability Extension includes Linux Virtual Server as the load-balancer, and SUSE OpenStack Cloud Crowbar uses HAProxy for this purpose (http://haproxy.1wt.eu/).
Though it is possible to use the same cluster for all of the roles above, the recommended setup is to use three clusters and to deploy the roles as follows:
data
cluster:
database-server
and
rabbitmq-server
network
cluster:
neutron-network
(as the
neutron-network
role may result in heavy network
load and CPU impact)
services
cluster: all other roles listed
above (as they are related to API/schedulers)
SUSE OpenStack Cloud Crowbar does not support High Availability for the LBaaS service plug-in. Thus, failover of a neutron load-balancer to another node can only be configured manually by editing the database.
For setting up the clusters, some special requirements and recommendations apply. For details, refer to Section 2.6.5, “Cluster Requirements and Recommendations”.
Recovery of the Control Node(s) is done automatically by the cluster software: if one Control Node fails, Pacemaker will fail over the services to another Control Node. If a failed Control Node is repaired and rebuilt via Crowbar, it will be automatically configured to join the cluster. At this point Pacemaker will have the option to fail back services if required.
If a Compute Node fails, all VMs running on that node will go down. While it cannot protect against failures of individual VMs, a
High Availability setup for Compute Nodes helps to minimize VM downtime caused by
Compute Node failures. If the nova-compute
service or
libvirtd
fail on a Compute Node, Pacemaker will
try to automatically recover them. If recovery fails, or the
node itself should become unreachable, the node will be fenced and the
VMs will be moved to a different Compute Node.
If you decide to use High Availability for Compute Nodes, your Compute Node will
be run as Pacemaker remote nodes. With the pacemaker-remote
service, High Availability clusters can be extended to control remote nodes without any
impact on scalability, and without having to install the full cluster stack
(including corosync
) on the remote nodes. Instead, each
Compute Node only runs the pacemaker-remote
service. The service
acts as a proxy, allowing the cluster stack on the “normal”
cluster nodes to connect to it and to control services remotely. Thus, the
node is effectively integrated into the cluster as a remote node. In this way,
the services running on the OpenStack compute nodes can be controlled from the core
Pacemaker cluster in a lightweight, scalable fashion.
Find more information about the pacemaker_remote
service in
Pacemaker Remote—Extending High Availability into
Virtual Nodes,
available at http://www.clusterlabs.org/doc/.
To configure High Availability for Compute Nodes, you need to adjust the following barclamp proposals:
Pacemaker—for details, see Section 12.2, “Deploying Pacemaker (Optional, HA Setup Only)”.
Nova—for details, see Section 12.11.1, “HA Setup for Nova”.
SUSE OpenStack Cloud Crowbar offers two different types of storage that can be used for the Storage Nodes: object storage (provided by the OpenStack Swift component) and block storage (provided by Ceph).
Both already consider High Availability aspects by design, therefore it does not require much effort to make the storage highly available.
The OpenStack Object Storage replicates the data by design, provided the following requirements are met:
The option 3
, the tested and recommended
value.
The number of Storage Nodes needs to be greater than the value set in the
option.
To avoid single points of failure, assign the
swift-storage
role to multiple nodes.
To make the API highly available, assign the
swift-proxy
role to a cluster instead of
assigning it to a single Control Node. See
Section 2.6.2.1, “Control Node(s)—Avoiding Points of Failure”. Other swift roles
must not be deployed on a cluster.
Ceph is a distributed storage solution that can provide High Availability. For High Availability redundant storage and monitors need to be configured in the Ceph cluster. For more information refer to the SUSE Enterprise Storage documentation at https://documentation.suse.com/ses/5.5/.
When considering setting up one or more High Availability clusters, refer to the chapter System Requirements in the High Availability Guide for SUSE Linux Enterprise High Availability Extension. The guide is available at https://documentation.suse.com/sle-ha/12-SP5/.
The HA requirements for Control Node also apply to SUSE OpenStack Cloud Crowbar. Note that by buying SUSE OpenStack Cloud Crowbar, you automatically get an entitlement for SUSE Linux Enterprise High Availability Extension.
Especially note the following requirements:
Each cluster needs to consist of at least three cluster nodes.
The Galera cluster needs an odd number of cluster nodes with a minimum of three nodes.
A cluster needs Quorum to keep services running. A three-node cluster can tolerate failure of only one node at a time, whereas a five-node cluster can tolerate failures of two nodes.
The cluster software will shut down “misbehaving” nodes
in a cluster to prevent them from causing trouble. This mechanism is
called fencing
or
STONITH.
A cluster without STONITH is not supported.
For a supported HA setup, ensure the following:
Each node in the High Availability cluster needs to have at least one STONITH device (usually a hardware device). We strongly recommend multiple STONITH devices per node, unless STONITH Block Device (SBD) is used.
The global cluster options stonith-enabled
and startup-fencing
must be set to
true
. These options are set automatically when
deploying the Pacemaker
barclamp. When you change them, you will lose support.
When deploying the Pacemaker
service, select a
STONITH: Configuration mode for STONITH
that matches your setup. If your STONITH devices support the
IPMI protocol, choosing the IPMI option is the easiest way to
configure STONITH. Another alternative is SBD. It provides a way to enable STONITH and fencing
in clusters without external power switches, but it requires shared
storage. For SBD requirements, see
http://linux-ha.org/wiki/SBD_Fencing, section
Requirements.
For more information, refer to the High Availability Guide, available at https://documentation.suse.com/sle-ha/12-SP5/. Especially read the following chapters: Configuration and Administration Basics, and Fencing and STONITH, Storage Protection.
For a supported HA setup, it is required to set up cluster communication via two or more redundant paths. For this purpose, use network device bonding and team network mode in your Crowbar network setup. For details, see Section 2.1.2.3, “Team Network Mode”. At least two Ethernet cards per cluster node are required for network redundancy. We advise using team network mode everywhere (not only between the cluster nodes) to ensure redundancy.
For more information, refer to the High Availability Guide, available at https://documentation.suse.com/sle-ha/12-SP5/. Especially read the following chapter: Network Device Bonding.
Using a second communication channel (ring) in Corosync (as an
alternative to network device bonding) is not supported yet in SUSE OpenStack Cloud Crowbar.
By default, SUSE OpenStack Cloud Crowbar uses the admin network (typically
eth0
) for the first Corosync ring.
The corosync
network communication layer is
crucial to the health of the cluster. corosync
traffic always
goes over the admin network.
Use redundant communication paths for the corosync
network communication layer.
Do not place the corosync
network communication layer
on interfaces shared with any other networks that could experience heavy
load, such as the OpenStack public / private / SDN / storage networks.
Similarly, if SBD over iSCSI is used as a STONITH device (see STONITH), do not place the iSCSI traffic on interfaces that could experience heavy load, because this might disrupt the SBD mechanism.
When using SBD as STONITH device, additional requirements apply for the shared storage. For details, see http://linux-ha.org/wiki/SBD_Fencing, section Requirements.
For a basic understanding and detailed information on the SUSE Linux Enterprise High Availability Extension (including the Pacemaker cluster stack), read the High Availability Guide. It is available at https://documentation.suse.com/sle-ha/12-SP5/.
In addition to the chapters mentioned in Section 2.6.5, “Cluster Requirements and Recommendations”, the following chapters are especially recommended:
Product Overview
Configuration and Administration Basics
The High Availability Guide also provides comprehensive information about the cluster management tools with which you can view and check the cluster status in SUSE OpenStack Cloud Crowbar. They can also be used to look up details like configuration of cluster resources or global cluster options. Read the following chapters for more information:
HA Web Console: Configuring and Managing Cluster Resources (Web Interface)
crm.sh
: Configuring and Managing
Cluster Resources (Command Line)
As outlined above, there are some important considerations to be made before deploying SUSE OpenStack Cloud. The following briefly summarizes what was discussed in detail in this chapter. Keep in mind that as of SUSE OpenStack Cloud Crowbar 8 it is not possible to change some aspects such as the network setup when SUSE OpenStack Cloud is deployed!
If you do not want to stick with the default networks and addresses, define custom networks and addresses. You need five different networks. If you need to separate the admin and the BMC network, a sixth network is required. See Section 2.1, “Network” for details. Networks that share interfaces need to be configured as VLANs.
The SUSE OpenStack Cloud networks are completely isolated, therefore it is not required to use public IP addresses for them. A class C network as used in this documentation may not provide enough addresses for a cloud that is supposed to grow. You may alternatively choose addresses from a class B or A network.
Determine how to allocate addresses from your network. Make sure not to allocate IP addresses twice. See Section 2.1.1, “Network Address Allocation” for the default allocation scheme.
Define which network mode to use. Keep in mind that all machines within the cloud (including the Administration Server) will be set up with the chosen mode and therefore need to meet the hardware requirements. See Section 2.1.2, “Network Modes” for details.
Define how to access the admin and BMC network(s): no access from the outside (no action is required), via an external gateway (gateway needs to be provided), or via bastion network. See Section 2.1.3, “Accessing the Administration Server via a Bastion Network” for details.
Provide a gateway to access the public network (public, nova-floating).
Make sure the Administration Server's host name is correctly configured
(hostname
-f
needs to return a
fully qualified host name). If this is not the case, run › › and add a fully qualified
host name.
Prepare a list of MAC addresses and the intended use of the corresponding host for all OpenStack nodes.
Depending on your network setup you have different options for providing up-to-date update repositories for SUSE Linux Enterprise Server and SUSE OpenStack Cloud for SUSE OpenStack Cloud deployment: using an existing SMT or SUSE Manager server, installing SMT on the Administration Server, synchronizing data with an existing repository, mounting remote repositories, or using physical media. Choose the option that best matches your needs.
Decide whether you want to deploy the object storage service Swift. If so, you need to deploy at least two nodes with sufficient disk space exclusively dedicated to Swift.
Decide which back-end to use with Cinder. If using the
cinder-volume
. Ceph needs a minimum of four exclusive nodes with sufficient disk space.
Make sure all Compute Nodes are equipped with sufficient hard disk space.
Decide whether to use different SSL certificates for the services and the API, or whether to use a single certificate.
Get one or more SSL certificates certified by a trusted third party source.
Make sure the hardware requirements for the different node types are met.
Make sure to have all required software at hand.
Deploying and installing SUSE OpenStack Cloud Crowbar is a multi-step process. Start by deploying a basic SUSE Linux Enterprise Server installation and the SUSE OpenStack Cloud Crowbar add-on product to the Administration Server. Then the product and update repositories need to be set up and the SUSE OpenStack Cloud network needs to be configured. Next, complete the Administration Server setup. After the Administration Server is ready, you can start deploying and configuring the OpenStack nodes. The complete node deployment is done automatically via Crowbar and Chef from the Administration Server. All you need to do is to boot the nodes using PXE and to deploy the OpenStack components to them.
Install SUSE Linux Enterprise Server 12 SP3 on the Administration Server with the add-on product SUSE OpenStack Cloud. Optionally select the Subscription Management Tool (SMT) pattern for installation. See Chapter 3, Installing the Administration Server.
Optionally set up and configure the SMT server on the Administration Server. See Chapter 4, Installing and Setting Up an SMT Server on the Administration Server (Optional).
Make all required software repositories available on the Administration Server. See Chapter 5, Software Repository Setup.
Set up the network on the Administration Server. See Chapter 6, Service Configuration: Administration Server Network Configuration.
Perform the Crowbar setup to configure the SUSE OpenStack Cloud network and to make the repository locations known. When the configuration is done, start the SUSE OpenStack Cloud Crowbar installation. See Chapter 7, Crowbar Setup.
Boot all nodes onto which the OpenStack components should be deployed using PXE and allocate them in the Crowbar Web interface to start the automatic SUSE Linux Enterprise Server installation. See Chapter 11, Installing the OpenStack Nodes.
Configure and deploy the OpenStack components via the Crowbar Web interface or command line tools. See Chapter 12, Deploying the OpenStack Services.
When all OpenStack components are up and running, SUSE OpenStack Cloud is ready. The cloud administrator can now upload images to enable users to start deploying instances. See the Administrator Guide and the Supplement to Administrator Guide and End User Guide.
In this chapter you will learn how to install the Administration Server from scratch. It will run on SUSE Linux Enterprise Server 12 SP3 and include the SUSE OpenStack Cloud Crowbar extension and, optionally, the Subscription Management Tool (SMT) server. Prior to starting the installation, refer to Section 2.4, “Hardware Requirements” and Section 2.5, “Software Requirements”.
One way to provide the repositories needed to set up the nodes in SUSE OpenStack Cloud is to install a Subscription Management Tool (SMT) server on the Administration Server, and then mirror all repositories from SUSE Customer Center via this server. Installing an SMT server on the Administration Server is optional. If your organization already provides an SMT server or a SUSE Manager server that can be accessed from the Administration Server, skip this step.
Nodes in SUSE OpenStack Cloud are automatically installed from the Administration Server. For this to happen, software repositories containing products, extensions, and the respective updates for all software need to be available on or accessible from the Administration Server. In this configuration step, these repositories are made available. There are two types of repositories:
Product Media Repositories: Product media repositories are copies of the installation media. They need to be directly copied to the Administration Server, “loop-mounted” from an iso image, or mounted from a remote server via NFS. Affected are SUSE Linux Enterprise Server 12 SP3 and SUSE OpenStack Cloud Crowbar 8. These are static repositories; they do not change or receive updates. See Section 5.1, “Copying the Product Media Repositories” for setup instructions.
Update and Pool Repositories: Update and Pool repositories are provided by the SUSE Customer Center. They contain all updates and patches for the products and extensions. To make them available for SUSE OpenStack Cloud they need to be mirrored from the SUSE Customer Center. Since their content is regularly updated, they must be kept in synchronization with SUSE Customer Center. For these purposes, SUSE provides either the Subscription Management Tool (SMT) or the SUSE Manager.
Prior to starting the SUSE OpenStack Cloud Crowbar installation, make sure the first network
interface (eth0
) gets a
fixed IP address from the admin network. A host and domain name
also need to be provided. Other interfaces will be automatically
configured during the SUSE OpenStack Cloud Crowbar installation.
The YaST Crowbar module enables you to configure all networks within the
cloud, to set up additional repositories, and to manage the Crowbar users.
This module should be launched before starting the SUSE OpenStack Cloud Crowbar installation. To start
this module, either run yast crowbar
or › › .
The last step in configuring the Administration Server is starting Crowbar.
In large deployments with many nodes, there are always some nodes that are in a fail or unknown state. New barclamps cannot be applied to them and values cannot be updated in some barclamps that are already deployed. This happens because Crowbar will refuse to apply a barclamp to a list of nodes if …
In this chapter you will learn how to install the Administration Server from scratch. It will run on SUSE Linux Enterprise Server 12 SP3 and include the SUSE OpenStack Cloud Crowbar extension and, optionally, the Subscription Management Tool (SMT) server. Prior to starting the installation, refer to Section 2.4, “Hardware Requirements” and Section 2.5, “Software Requirements”.
Start the installation by booting into the SUSE Linux Enterprise Server 12 SP3 installation system. For detailed installation instructions for SUSE Linux Enterprise Server, refer to https://documentation.suse.com/sles/12-SP5/single-html/SLES-deployment/#cha-install.
The following sections will only cover the differences from the default installation process.
Registering SUSE Linux Enterprise Server 12 SP3 during the installation process is required for getting product updates and for installing the SUSE OpenStack Cloud Crowbar extension. Refer to https://documentation.suse.com/sles/12-SP5/single-html/SLES-deployment/#sec-i-yast2-conf-manual-cc
After a successful registration you will be asked whether to add the update repositories. If you agree, the latest updates will automatically be installed, ensuring that your system is on the latest patch level after the initial installation. We strongly recommend adding the update repositories immediately. If you choose to skip this step you need to perform an online update later, before starting the SUSE OpenStack Cloud Crowbar installation.
To register a product, you need to have a SUSE login. If you do not have such a login, create it at http://www.suse.com/.
SUSE OpenStack Cloud is an extension to SUSE Linux Enterprise Server. Installing it during the SUSE Linux Enterprise Server installation is the easiest and recommended way to set up the Administration Server. To get access to the extension selection dialog, you need to register SUSE Linux Enterprise Server 12 SP3 during the installation. After a successful registration, the SUSE Linux Enterprise Server 12 SP3 installation continues with the . Choose and provide the registration key you obtained by purchasing SUSE OpenStack Cloud Crowbar. The registration and the extension installation require an Internet connection.
Alternatively, install the SUSE OpenStack Cloud Crowbar after the SUSE Linux Enterprise Server 12 SP3 installation via › › . For details, refer to the section https://documentation.suse.com/sles/12-SP5/single-html/SLES-deployment/#sec-add-ons-extensions.
Currently, Crowbar requires /opt
to be writable. We recommend creating a separate partition
or volume formatted with XFS for /srv
with a size of
at least 30 GB.
The default file system on SUSE Linux Enterprise Server 12 SP3 is Btrfs with snapshots enabled.
SUSE OpenStack Cloud Crowbar installs into /opt
, a directory that is
excluded from snapshots. Reverting to a snapshot may therefore break the
SUSE OpenStack Cloud Crowbar installation. We recommend disabling Btrfs snapshots on
the Administration Server.
Help on using the partitioning tool is available at the section https://documentation.suse.com/sles/12-SP5/single-html/SLES-deployment/#sec-yast2-i-y2-part-expert.
In the final installation step, https://documentation.suse.com/sles/12-SP5/single-html/SLES-deployment/#sec-i-yast2-proposal.
, you need to adjust the software selection and the firewall settings for your Administration Server setup. For more information refer to theInstalling a minimal base system is sufficient to set up the Administration Server. The following patterns are the minimum required:
SUSE OpenStack Cloud Crowbar Extension)
(in case you have chosen to install theIf you do not have a SUSE Manager or SMT server in your organization, or are planning to manually update the repositories required for deployment of the SUSE OpenStack Cloud nodes, you need to set up an SMT server on the Administration Server. Choose the pattern in addition to the patterns listed above to install the SMT server software.
SUSE OpenStack Cloud Crowbar requires disabling the firewall on the Administration Server. You can disable the firewall during installation in the section. If your environment requires a firewall to be active at this stage of the installation, you can disable the firewall during your final network configuration (see Chapter 6, Service Configuration: Administration Server Network Configuration). Optionally, you can also enable SSH access to the Administration Server in this section.
Setting HTTP_PROXY without properly configuring NO_PROXY for the Administration Server might result in chef-client failing in non-obvious ways.
One way to provide the repositories needed to set up the nodes in SUSE OpenStack Cloud is to install a Subscription Management Tool (SMT) server on the Administration Server, and then mirror all repositories from SUSE Customer Center via this server. Installing an SMT server on the Administration Server is optional. If your organization already provides an SMT server or a SUSE Manager server that can be accessed from the Administration Server, skip this step.
When installing an SMT server on the Administration Server, use it exclusively for SUSE OpenStack Cloud Crowbar. To use the SMT server for other products, run it outside of SUSE OpenStack Cloud Crowbar. Make sure it can be accessed from the Administration Server for mirroring the repositories needed for SUSE OpenStack Cloud Crowbar.
When the SMT server is installed on the Administration Server, Crowbar
provides the mirrored repositories on port 8091
.
If you have not installed the SMT server during the initial Administration Server installation as suggested in Section 3.5.1, “Software Selection”, run the following command to install it:
sudo zypper in -t pattern smt
No matter whether the SMT server was installed during the initial installation or in the running system, it needs to be configured with the following steps.
To configure the SMT server, a SUSE account is required. If you do not have such an account, register at http://www.suse.com/. All products and extensions for which you want to mirror updates with the SMT server should be registered at the SUSE Customer Center (http://scc.suse.com/).
Configuring the SMT server requires you to have your mirroring credentials (user name and password) and your registration e-mail address at hand. To access them, proceed as follows:
Open a Web browser and log in to the SUSE Customer Center at http://scc.suse.com/.
Click your name to see the e-mail address which you have registered.
Click
› to obtain your mirroring credentials (user name and password).Start
› › .Activate
.Enter the
data as follows:not activate this option | : Do
: The user name you retrieved from the SUSE Customer Center |
: The password you retrieved from the SUSE Customer Center |
Check your input with success
, check the credentials you entered.
Enter the e-mail address you retrieved from the SUSE Customer Center at
.shows the HTTP address of your server. Usually it should not be necessary to change it.
Select
to proceed to step two of the .Enter a
and confirm it by entering it once again.Enter one or more e-mail addresses to which SMT status reports are sent by selecting
.Select
to save your SMT configuration. When setting up the database you will be prompted for the MariaDB root password. If you have not already created one then create it in this step. Note that this is the global MariaDB root password, not the database password for the SMT user you specified before.
The SMT server requires a server certificate at
/etc/pki/trust/anchors/YaST-CA.pem
. Choose
, provide a password and choose
to create such a certificate. If your
organization already provides a CA certificate,
this step and import the certificate via › › after the SMT
configuration is done. See
https://documentation.suse.com/sles/12-SP5/single-html/SLES-security/#cha-security-yast-ca
for more information.
After you complete your configuration a synchronization check with the SUSE Customer Center will run, which may take several minutes.
The final step in setting up the SMT server is configuring it to mirror the repositories needed for SUSE OpenStack Cloud. The SMT server mirrors the repositories from the SUSE Customer Center. Make sure to have the appropriate subscriptions registered in SUSE Customer Center with the same e-mail address you specified when configuring SMT. For details on the required subscriptions refer to Section 2.5, “Software Requirements”.
Mirroring the SUSE Linux Enterprise Server 12 SP3 and SUSE OpenStack Cloud Crowbar 8
repositories is mandatory. Run the following commands as user
root
to add them to the list of mirrored repositories:
for REPO in SLES12-SP3-{Pool,Updates} SUSE-OpenStack-Cloud-Crowbar-8-{Pool,Updates}; do
smt-repos $REPO sle-12-x86_64 -e
done
The following optional repositories provide high availability and storage:
For the optional HA setup you need to mirror the SLE12-HA12-SP3
repositories. Run the following commands as user root
to add
them to the list of mirrored repositories:
for REPO in SLE12-HA12-SP3-{Pool,Updates}; do smt-repos $REPO sle-12-x86_64 -e done
The SUSE Enterprise Storage repositories are needed if you plan to use an external
Ceph with SUSE OpenStack Cloud. Run the following commands as user root
to
add them to the list of mirrored repositories:
for REPO in SUSE-Enterprise-Storage-5-{Pool,Updates}; do smt-repos $REPO sle-12-x86_64 -e done
New repositories added to SMT must be updated immediately by running the following command as user root
:
smt-mirror -L /var/log/smt/smt-mirror.log
This command will download several GB of patches. This process may last
up to several hours. A log file is written to
/var/log/smt/smt-mirror.log
. After this first manual update the repositories are updated automatically via cron
job. A list of all
repositories and their location in the file system on the Administration Server can be
found at Table 5.2, “SMT Repositories Hosted on the Administration Server”.
For detailed information about SMT refer to the Subscription Management Tool manual at https://documentation.suse.com/sles/12-SP5/single-html/SLES-smt/.
Nodes in SUSE OpenStack Cloud are automatically installed from the Administration Server. For this to happen, software repositories containing products, extensions, and the respective updates for all software need to be available on or accessible from the Administration Server. In this configuration step, these repositories are made available. There are two types of repositories:
Product Media Repositories: Product media repositories are copies of the installation media. They need to be directly copied to the Administration Server, “loop-mounted” from an iso image, or mounted from a remote server via NFS. Affected are SUSE Linux Enterprise Server 12 SP3 and SUSE OpenStack Cloud Crowbar 8. These are static repositories; they do not change or receive updates. See Section 5.1, “Copying the Product Media Repositories” for setup instructions.
Update and Pool Repositories: Update and Pool repositories are provided by the SUSE Customer Center. They contain all updates and patches for the products and extensions. To make them available for SUSE OpenStack Cloud they need to be mirrored from the SUSE Customer Center. Since their content is regularly updated, they must be kept in synchronization with SUSE Customer Center. For these purposes, SUSE provides either the Subscription Management Tool (SMT) or the SUSE Manager.
The files in the product repositories for SUSE Linux Enterprise Server and SUSE OpenStack Cloud do not
change, therefore they do not need to be synchronized with a remote
source. It is sufficient to either copy the data (from a remote host or
the installation media), to mount the product repository from a remote
server via NFS
, or to loop mount a copy of the
installation images.
Note that the SUSE Linux Enterprise Server product repository must be directly available from the local directory listed below. It is not possible to use a symbolic link to a directory located elsewhere, since this will cause booting via PXE to fail.
The SUSE Linux Enterprise Server product repositories need to be available locally
to enable booting via PXE for node deployment. The SUSE OpenStack Cloud Crowbar
repository may also be served via HTTP from a remote
host. In this case, enter the URL to the Cloud
repository as described in Section 7.4, “.
”
We recommend copying the data to the Administration Server as the best solution. It does not require much hard disk space (approximately 900 MB). Nor does it require the Administration Server to access a remote host from a different network.
The following product media must be copied to the specified directories:
Repository |
Directory |
---|---|
SUSE Linux Enterprise Server 12 SP3 DVD #1 |
|
SUSE OpenStack Cloud Crowbar 8 DVD #1 |
|
The data can be copied by a variety of methods:
We recommended using rsync
for copying. If the
installation data is located on a removable device, make sure to mount
it first (for example, after inserting the DVD1 in the Administration Server and
waiting for the device to become ready):
mkdir -p /srv/tftpboot/suse-12.3/x86_64/install mount /dev/dvd /mnt rsync -avP /mnt/ /srv/tftpboot/suse-12.3/x86_64/install/ umount /mnt
mkdir -p /srv/tftpboot/suse-12.3/x86_64/repos/Cloud mount /dev/dvd /mnt rsync -avP /mnt/ /srv/tftpboot/suse-12.3/x86_64/repos/Cloud/ umount /mnt
If the data is provided by a remote machine, log in to that machine and
push the data to the Administration Server (which has the IP address 192.168.124.10
in the following
example):
mkdir -p /srv/tftpboot/suse-12.3/x86_64/install rsync -avPz /data/SLES-12-SP3/DVD1/ 192.168.124.10:/srv/tftpboot/suse-12.3/x86_64/install/
mkdir -p /srv/tftpboot/suse-12.3/x86_64/repos/Cloud rsync -avPz /data/SUSE-OPENSTACK-CLOUD//DVD1/ 192.168.124.10:/srv/tftpboot/suse-12.3/x86_64/repos/Cloud/
If the installation data is provided via NFS by a remote machine, mount
the respective shares as follows. To automatically mount these
directories either create entries in /etc/fstab
or
set up the automounter.
mkdir -p /srv/tftpboot/suse-12.3/x86_64/install mount -t nfs nfs.example.com:/exports/SLES-12-SP3/x86_64/DVD1/ /srv/tftpboot/suse-12.3/x86_64/install
mkdir -p /srv/tftpboot/suse-12.3/x86_64/repos/Cloud/ mount -t nfs nfs.example.com:/exports/SUSE-OPENSTACK-CLOUD/DVD1/ /srv/tftpboot/suse-12.3/x86_64/repos/Cloud
The product repositories can also be made available by copying the
respective ISO images to the Administration Server and mounting them. To
automatically mount these directories either create entries in
/etc/fstab
or set up the automounter.
mkdir -p /srv/tftpboot/suse-12.3/x86_64/install/ mount -o loop /local/SLES-12-SP3-x86_64-DVD1.iso /srv/tftpboot/suse-12.3/x86_64/install
mkdir -p /srv/tftpboot/suse-12.3/x86_64/repos/Cloud/
mount -o loop /local/SUSE-OPENSTACK-CLOUD-8-x86_64-DVD1.iso /srv/tftpboot/suse-12.3/x86_64/repos/Cloud
Update and Pool Repositories are required on the Administration Server to set up and maintain the SUSE OpenStack Cloud nodes. They are provided by SUSE Customer Center and contain all software packages needed to install SUSE Linux Enterprise Server 12 SP3 and the extensions (pool repositories). In addition, they contain all updates and patches (update repositories). Update repositories are used when deploying the nodes that build SUSE OpenStack Cloud to ensure they are initially equipped with the latest software versions available.
The repositories can be made available on the Administration Server using one or more of the following methods:
When all update and pool repositories are managed by an SMT server installed on the Administration Server (see Chapter 4, Installing and Setting Up an SMT Server on the Administration Server (Optional)), make sure the repository location in YaST Crowbar is set to (this is the default). For details, see Section 7.4, “. No further action is required. The SUSE ”OpenStack Cloud Crowbar installation automatically detects all available repositories.
To use repositories from a remote SMT server, you first need to make sure all required repositories are mirrored on the server. Refer to Section 4.3, “Setting up Repository Mirroring on the SMT Server” for more information. When all update and pool repositories are managed by a remote SMT server, make sure the repository location in YaST Crowbar is set to . For details, see Section 7.4, “. No further action is required. The SUSE ”OpenStack Cloud Crowbar installation automatically detects all available repositories.
When using an external SMT server, it needs to be reachable by all nodes. This means that the SMT server either needs to be part of the admin network or it needs to be accessible via the default route of the nodes. The latter can be either the gateway of the admin network or the gateway of the public network.
To use repositories from SUSE Manager you first need to make sure all required products and extensions are registered, and the corresponding channels are mirrored in SUSE Manager (refer to Table 5.4, “SUSE Manager Repositories (Channels)” for a list of channels).
An external SUSE Manager server needs to be accessible to all nodes in SUSE OpenStack Cloud. The network hosting the SUSE Manager server must be added to the network definitions as described in Section 7.5.8, “Providing Access to External Networks”.
By default SUSE Manager does not expose repositories for direct access. To access them via HTTPS, you need to create a SUSE OpenStack Cloud Crowbar, SLES High Availability Extension and SUSE Enterprise Storage). Instructions for creating a distribution are in the SUSE Manager documentation in https://documentation.suse.com/suma/.
for auto-installation for the SUSE Linux Enterprise Server 12 SP3 (x86_64) product. Creating this distribution makes the update repositories for this product available, including the repositories for all registered add-on products (like
During the distribution setups you need to provide a
Table 5.4, “SUSE Manager Repositories (Channels)” we
assume the following label has been provided:
sles12-sp3-x86_64
.
When all update and pool repositories are managed by a SUSE Manager server, make sure the repository location in YaST Crowbar is set to Section 7.4, “. No further action is required. The SUSE ”OpenStack Cloud Crowbar installation automatically detects all available repositories.
. For details, seeThe autoinstallation tree provided by SUSE Manager does not provide the SLES Pool repository. Although this repository is not used for node installation, it needs to be present. To work around this issue, it is sufficient to create an empty Pool repository for SUSE Linux Enterprise Server 12 SP3:
mkdir /srv/tftpboot/suse-12.3/x86_64/repos/SLES12-SP3-Pool/ createrepo /srv/tftpboot/suse-12.3/x86_64/repos/SLES12-SP3-Pool/
If you want to keep your SUSE OpenStack Cloud network as isolated from the company network as possible, or your infrastructure does not allow accessing a SUSE Manager or an SMT server, you can alternatively provide access to the required repositories by one of the following methods:
Mount the repositories from a remote server.
Synchronize the repositories from a remote server (for example via
rsync
and cron).
Manually synchronize the update repositories from removable media.
We strongly recommended making the repositories available at the default locations on the Administration Server as listed in Table 5.5, “Default Repository Locations on the Administration Server”. When choosing these locations, it is sufficient to set the repository location in YaST Crowbar to . You do not need to specify a detailed location for each repository. Refer to Section 7.4, “ for details. If you prefer to use different locations, you need to announce each location with YaST Crowbar. ”
During the installation of the Administration Server, repository locations for SUSE Linux Enterprise Server 12 SP3 are automatically added to the Administration Server. They point to the source used to install the Administration Server and to the SUSE Customer Center. These repository locations have no influence on the repositories used to set up nodes in the cloud. They are solely used to maintain and update the Administration Server itself.
However, as the Administration Server and all nodes in the cloud use the same operating system—SUSE Linux Enterprise Server 12 SP3—it makes sense to use the same repositories for the cloud and the Administration Server. To avoid downloading the same patches twice, change this setup so that the repositories set up for SUSE OpenStack Cloud deployment are also used on the Administration Server.
To do so, you need to disable or delete all services. In a second step all SUSE Linux Enterprise Server and SUSE OpenStack Cloud repositories need to be edited to point to the alternative sources. Use either Zypper or YaST to edit the repository setup. Note that changing the repository setup on the Administration Server is optional.
The following tables show the locations of all repositories that can be used for SUSE OpenStack Cloud.
Repository |
Directory |
---|---|
Mandatory Repositories | |
SLES12-SP3-Pool |
|
SLES12-SP3-Updates |
|
SUSE-OpenStack-Cloud-Crowbar-8-Pool |
|
SUSE-OpenStack-Cloud-Crowbar-8-Updates |
|
Optional Repositories | |
SLE12-HA12-SP3-Pool |
|
SLE12-HA12-SP3-Updates |
|
SUSE-Enterprise-Storage-5-Pool |
|
SUSE-Enterprise-Storage-5-Updates |
|
Repository |
URl |
---|---|
Mandatory Repositories | |
SLES12-SP3-Pool |
http://smt.example.com/repo/SUSE/Products/SLE-SERVER/12-SP3/x86_64/product/ |
SLES12-SP3-Updates |
http://smt.example.com/repo/SUSE/Updates/SLE-SERVER/12-SP3/x86_64/update/ |
SUSE-OpenStack-Cloud-Crowbar-8-Pool |
http://smt.example.com/repo/SUSE/Products/OpenStack-Cloud/7/x86_64/product/ |
SUSE-OpenStack-Cloud-Crowbar-8-Updates |
http://smt.example.com/repo/SUSE/Updates/OpenStack-Cloud/7/x86_64/update/ |
Optional Repositories | |
SLE12-HA12-SP3-Pool |
http://smt.example.com/repo/SUSE/Products/SLE-HA/12-SP3/x86_64/product/ |
SLE12-HA12-SP3-Updates |
http://smt.example.com/repo/SUSE/Updates/SLE-HA/12-SP3/x86_64/update/ |
SUSE-Enterprise-Storage-5-Pool |
http://smt.example.com/repo/SUSE/Products/Storage/4/x86_64/product/ |
SUSE-Enterprise-Storage-5-Updates |
http://smt.example.com/repo/SUSE/Updates/Storage/4/x86_64/update/ |
Repository |
URL |
---|---|
Mandatory Repositories | |
SLES12-SP3-Updates |
http://manager.example.com/ks/dist/child/sles12-sp3-updates-x86_64/sles12-sp3-x86_64/ |
SUSE-OpenStack-Cloud-Crowbar-8-Pool |
http://manager.example.com/ks/dist/child/suse-openstack-cloud-7-pool-x86_64/sles12-sp3-x86_64/ |
SUSE-OpenStack-Cloud-Crowbar-8--Updates |
http://manager.example.com/ks/dist/child/suse-openstack-cloud-7-updates-x86_64/sles12-sp3-x86_64/ |
Optional Repositories | |
SLE12-HA12-SP3-Pool |
http://manager.example.com/ks/dist/child/sle-ha12-sp3-pool-x86_64/sles12-sp3-x86_64/ |
SLE12-HA12-SP3-Updates |
http://manager.example.com/ks/dist/child/sle-ha12-sp3-updates-x86_64/sles12-sp3-x86_64/ |
SUSE-Enterprise-Storage-5-Pool |
http://manager.example.com/ks/dist/child/suse-enterprise-storage-2.1-pool-x86_64/sles12-sp3-x86_64/ |
SUSE-Enterprise-Storage-5-Updates |
http://manager.example.com/ks/dist/child/suse-enterprise-storage-4-updates-x86_64/sles12-sp3-x86_64/ |
The following table shows the recommended default repository locations to use when manually copying, synchronizing, or mounting the repositories. When choosing these locations, it is sufficient to set the repository location in YaST Crowbar to Section 5.2.4, “Alternative Ways to Make the Repositories Available” and Section 7.4, “ for details. ”
. You do not need to specify a detailed location for each repository. Refer to
Channel |
Directory on the Administration Server |
---|---|
Mandatory Repositories | |
SLES12-SP3-Pool |
|
SLES12-SP3-Updates |
|
SUSE-OpenStack-Cloud-Crowbar-8-Pool |
|
SUSE-OpenStack-Cloud-Crowbar-8-Updates |
|
Optional Repositories | |
SLE12-HA12-SP3-Pool |
|
SLE12-HA12-SP3-Updates |
|
SUSE-Enterprise-Storage-5-Pool |
|
SUSE-Enterprise-Storage-5-Updates |
|
Prior to starting the SUSE OpenStack Cloud Crowbar installation, make sure the first network
interface (eth0
) gets a
fixed IP address from the admin network. A host and domain name
also need to be provided. Other interfaces will be automatically
configured during the SUSE OpenStack Cloud Crowbar installation.
To configure the network interface proceed as follows:
Start
› › .
Switch to the eth0
and
choose .
Switch to the 192.168.124.10
and a network mask of
255.255.255.0
. Using a
different IP address requires adjusting the Crowbar configuration in a
later step as described in Chapter 7, Crowbar Setup.
Check the settings on the
tab. The device needs to be activated . Confirm your settings with .
Back on the 192.168.124.1
. Confirm your settings
with .
Choose
from the dialog and set the and . Examples in this book assume admin.cloud.example.com for the host/domain name.If the Administration Server has access to the outside, you can add additional name servers here that will automatically be used to forward requests. The Administration Server's name server will automatically be configured during the SUSE OpenStack Cloud Crowbar installation to forward requests for non-local records to those server(s).
Last, check if the firewall is disabled. Return to YaST's main menu (
) and start › . On › , the firewall needs to be disabled. Confirm your settings with .Setting up the SUSE OpenStack Cloud will also install a DNS server for all nodes in the cloud. The domain name you specify for the Administration Server will be used for the DNS zone. It is required to use a sub-domain such as cloud.example.com. See Section 2.1.4, “DNS and Host Names” for more information.
The host name and the FQDN need to be resolvable with
hostname
-f
. Double-check whether
/etc/hosts
contains an appropriate entry for the
Administration Server. It should look like the following:
192.168.124.10 admin.cloud.example.com admin
It is not possible to change the Administration Server host name or the FQDN after the SUSE OpenStack Cloud Crowbar installation has been completed.
The YaST Crowbar module enables you to configure all networks within the
cloud, to set up additional repositories, and to manage the Crowbar users.
This module should be launched before starting the SUSE OpenStack Cloud Crowbar installation. To start
this module, either run yast crowbar
or › › .
In this section, you can manage users for the Crowbar Web interface. The user
crowbar
(password
crowbar
) is preconfigured. Use the
, , and
buttons to manage user accounts. Users configured
here have no relation to existing system users on the Administration Server.
Use the Section 2.1, “Network”). Change the IP address assignment for each network under . You may also add a bridge ( ) or a VLAN ( , ) to a network. Only change the latter two settings if you really know what you require; we recommend sticking with the defaults.
tab to change the default network setup (described inAfter you have completed the SUSE OpenStack Cloud Crowbar installation, you cannot change the network setup. If you do need to change it, you must completely set up the Administration Server again.
As of SUSE OpenStack Cloud Crowbar 8, using a VLAN for the admin network is only supported on a native/untagged VLAN. If you need VLAN support for the admin network, it must be handled at switch level.
When changing the network configuration with YaST or by editing
/etc/crowbar/network.json
, you can define VLAN
settings for each network. For the networks nova-fixed
and nova-floating
, however, special rules apply:
nova-fixed: The setting will be ignored. However, VLANs will automatically be used if deploying Neutron with VLAN support (using the drivers linuxbridge, openvswitch plus VLAN, or cisco_nexus). In this case, you need to specify a correct for this network.
nova-floating: When using a VLAN for
nova-floating
(which is the default), the and settings for
and default to
the same.
You have the option of separating public and floating networks with a
custom configuration. Configure your own separate floating network (not as
a subnet of the public network), and give the floating network its own
router. For example, define nova-floating
as part of an
external network with a custom bridge-name
. When you are
using different networks and OpenVSwitch is configured, the pre-defined
bridge-name
won't work.
Other, more flexible network mode setups, can be configured by manually editing the Crowbar network configuration files. See Section 7.5, “Custom Network Configuration” for more information. SUSE or a partner can assist you in creating a custom setup within the scope of a consulting services agreement. See http://www.suse.com/consulting/ for more information on SUSE consulting.
If you want to separate the admin and the BMC network, you must change the settings for the networks
and . The is used to generate a VLAN tagged interface on the Administration Server that can access the network. The needs to be in the same ranges as , and needs to have enabled.
bmc |
bmc_vlan | |
---|---|---|
Subnet |
| |
Netmask |
| |
Router |
| |
Broadcast |
| |
Host Range |
|
|
VLAN |
yes | |
VLAN ID |
100 | |
Bridge |
no |
On the Section 2.1.2, “Network Modes” for details.
tab you can choose between , , and . In single mode, all traffic is handled by a single Ethernet card. Dual mode requires two Ethernet cards and separates traffic for private and public networks. SeeTeam mode is similar to single mode, except that you combine several Ethernet cards to a “bond”. It is required for an HA setup of SUSE OpenStack Cloud. When choosing this mode, you also need to specify a . This option lets you define whether to focus on reliability (fault tolerance), performance (load balancing), or a combination of both. You can choose from the following modes:
Default mode in SUSE OpenStack Cloud Crowbar. Packets are transmitted in round-robin fashion from the first to the last available interface. Provides fault tolerance and load balancing.
Only one network interface is active. If it fails, a different interface becomes active. This setting is the default for SUSE OpenStack Cloud. Provides fault tolerance.
Traffic is split between all available interfaces based on the following
policy: [(source MAC address XOR'd with destination MAC address
XOR packet type ID) modulo slave count]
Requires support from
the switch. Provides fault tolerance and load balancing.
All traffic is broadcast on all interfaces. Requires support from the switch. Provides fault tolerance.
Aggregates interfaces into groups that share the same speed and duplex
settings. Requires ethtool
support in the interface
drivers, and a switch that supports and is configured for IEEE 802.3ad
Dynamic link aggregation. Provides fault tolerance and load balancing.
Adaptive transmit load balancing. Requires ethtool
support in the interface drivers but no switch support. Provides fault
tolerance and load balancing.
Adaptive load balancing. Requires ethtool
support in
the interface drivers but no switch support. Provides fault tolerance and
load balancing.
For a more detailed description of the modes, see https://www.kernel.org/doc/Documentation/networking/bonding.txt.
The Section 2.1, “Network”, one way to access the Administration Server from a defined external network is via a Bastion network and a second network card (as opposed to providing an external gateway).
tab of the YaST Crowbar module also lets you set up a Bastion network. As outlined inTo set up the Bastion network, you need to have a static IP address for the Administration Server from the external network. The example configuration used below assumes that the external network from which to access the admin network has the following addresses. Adjust them according to your needs.
Subnet |
|
Netmask |
|
Broadcast |
|
Gateway |
|
Static Administration Server address |
|
In addition to the values above, you need to enter the Section 7.5.5, “Network Conduits” for details on the
syntax. The default value ?1g2
matches the second
interface (“eth1”) of the system.
After you have completed the SUSE OpenStack Cloud Crowbar installation, you cannot change the network setup. If you do need to change it, you must completely set up the Administration Server again.
The example configuration from above allows access to the SUSE OpenStack Cloud nodes
from within the bastion network. If you want to
access nodes from outside the bastion network, make the router for the
bastion network the default router for the Administration Server. This is achieved by
setting the value for the bastion network's entry to a lower value than the corresponding entry
for the admin network. By default no router preference is set for the
Administration Server—in this case, set the preference for the bastion network
to 5
.
If you use a Linux gateway between the outside and the bastion network, you also need to disable route verification (rp_filter) on the Administration Server. Do so by running the following command on the Administration Server:
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
That command disables route verification for the current session, so the
setting will not survive a reboot. Make it permanent by editing
/etc/sysctl.conf
and setting the value for
to 0
.
This dialog lets you announce the locations of the product, pool, and update repositories (see Chapter 5, Software Repository Setup for details). You can choose between four alternatives:
If you have an SMT server installed on the Administration Server as explained in Chapter 4, Installing and Setting Up an SMT Server on the Administration Server (Optional), choose this option. The repository details do not need to be provided as they will be configured automatically. This option will be applied by default if the repository configuration has not been changed manually.
If you use a remote SMT for all repositories,
choose this option and provide the (in the
form of http://smt.example.com
). The repository
details do not need to be provided, they will be configured
automatically.
If you use a remote SUSE Manager server for all
repositories, choose this option and provide the (in the form of
http://manager.example.com
).
If you use different sources for your repositories or are using
non-standard locations, choose this option and manually provide a
location for each repository. This can either be a local directory
(/srv/tftpboot/suse-12.3/x86_64/repos/SLES12-SP3-Pool/
)
or a remote location
(http://manager.example.com/ks/dist/child/sles12-sp3-updates-x86_64/sles12-sp3-x86_64/
).
Activating ensures that you will be
informed if a repository is not available during node deployment,
otherwise errors will be silently ignored.
The How to make custom software repositories from an external server (for example a remote SMT or SUSE M..? for instructions.
dialog allows adding additional repositories. SeeIf you have made the repositories available in the default locations on the Administration Server (see Table 5.5, “Default Repository Locations on the Administration Server” for a list), choose and leave the empty (default). The repositories will automatically be detected.
To adjust the pre-defined network setup of SUSE OpenStack Cloud beyond the scope of changing IP address assignments (as described in Chapter 7, Crowbar Setup), modify the network barclamp template.
The Crowbar network barclamp provides two functions for the system. The first
is a common role to instantiate network interfaces on the Crowbar managed
systems. The other function is address pool management. While the addresses
can be managed with the YaST Crowbar module, complex network setups require
to manually edit the network barclamp template file
/etc/crowbar/network.json
. This section explains the
file in detail. Settings in this file are applied to all nodes in SUSE OpenStack Cloud.
(See Section 7.5.11, “Matching Logical and Physical Interface Names with network-json-resolve” to learn how to verify
your correct network interface names.)
After you have completed the SUSE OpenStack Cloud Crowbar installation installation, you cannot change the network setup. If you do need to change it, you must completely set up the Administration Server again.
The only exception to this rule is the interface map, which can be changed after setup. See Section 7.5.3, “Interface Map” for details.
network.json
#
The network.json
file is located in
/etc/crowbar/
. The template has the following general
structure:
{ "attributes" : { "network" : { "mode" : "VALUE", "start_up_delay" : VALUE, "teaming" : { "mode": VALUE },1 "enable_tx_offloading" : VALUE, "enable_rx_offloading" : VALUE, "interface_map"2 : [ ... ], "conduit_map"3 : [ ... ], "networks"4 : { ... }, } } }
General attributes. Refer to Section 7.5.2, “Global Attributes” for details. | |
Interface map section. Defines the order in which the physical network interfaces are to be used. Refer to Section 7.5.3, “Interface Map” for details. | |
Network conduit section defining the network modes and the network interface usage. Refer to Section 7.5.5, “Network Conduits” for details. | |
Network definition section. Refer to Section 7.5.7, “Network Definitions” for details. |
The order in which the entries in the network.json
file appear may differ from the one listed above. Use your editor's search
function to find certain entries.
The most important options to define in the global attributes section are the default values for the network and bonding modes. The following global attributes exist:
{ "attributes" : { "network" : { "mode" : "single",1 "start_up_delay" : 30,2 "teaming" : { "mode": 5 },3 "enable_tx_offloading" : true, 4 "enable_rx_offloading" : true, 4 "interface_map" : [ ... ], "conduit_map" : [ ... ], "networks" : { ... }, } } }
Network mode. Defines the configuration name (or name space) to be used from the conduit_map (see Section 7.5.5, “Network Conduits”). Your choices are single, dual, or team. | |
Time (in seconds) the Chef-client waits for the network interfaces to come online before timing out. | |
Default bonding mode. For a list of available modes, see Section 7.3, “. ” | |
Turn on/off TX and RX checksum offloading. If set to
Checksum offloading is set to ![]() Important: Change of the Default Value
Starting with SUSE OpenStack Cloud Crowbar, the default value for TX and RX checksum
offloading changed from
To check which defaults a network driver uses, run
Note that if the output shows a value marked as
|
By default, physical network interfaces are used in the order they appear
under /sys/class/net/
. If you want to apply a
different order, you need to create an interface map where you can specify
a custom order of the bus IDs. Interface maps are created for specific
hardware configurations and are applied to all machines matching this
configuration.
{ "attributes" : { "network" : { "mode" : "single", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true , "enable_rx_offloading" : true , "interface_map" : [ { "pattern" : "PowerEdge R610"1, "serial_number" : "0x02159F8E"2, "bus_order" : [3 "0000:00/0000:00:01", "0000:00/0000:00:03" ] } ... ], "conduit_map" : [ ... ], "networks" : { ... }, } } }
Hardware specific identifier. This identifier can be obtained by running
the command | |
Additional hardware specific identifier. This identifier can be used in
case two machines have the same value for | |
Bus IDs of the interfaces. The order in which they are listed here
defines the order in which Chef addresses the interfaces. The IDs can
be obtained by listing the contents of
|
The physical interface used to boot the node via PXE must always be listed first.
Contrary to all other sections in network.json
, you
can change interface maps after completing the SUSE OpenStack Cloud Crowbar installation. However, nodes
that are already deployed and affected by these changes must be deployed
again. Therefore, we do not recommend making changes to the interface map
that affect active nodes.
If you change the interface mappings after completing the SUSE OpenStack Cloud Crowbar installation you
must not make your changes by editing
network.json
. You must rather use the Crowbar Web
interface and open › › › . Activate your changes by clicking
.
Get the machine identifier by running the following command on the machine to which the map should be applied:
~ # dmidecode -s system-product-name AS 2003R
The resulting string needs to be entered on the
http://www.ruby-doc.org/core-2.0/Regexp.html for a
reference). Unless the pattern starts with ^
and ends
with $
, a substring match is performed against the
name returned from the above commands.
List the interface devices in /sys/class/net
to get
the current order and the bus ID of each interface:
~ # ls -lgG /sys/class/net/ | grep eth lrwxrwxrwx 1 0 Jun 19 08:43 eth0 -> ../../devices/pci0000:00/0000:00:1c.0/0000:09:00.0/net/eth0 lrwxrwxrwx 1 0 Jun 19 08:43 eth1 -> ../../devices/pci0000:00/0000:00:1c.0/0000:09:00.1/net/eth1 lrwxrwxrwx 1 0 Jun 19 08:43 eth2 -> ../../devices/pci0000:00/0000:00:1c.0/0000:09:00.2/net/eth2 lrwxrwxrwx 1 0 Jun 19 08:43 eth3 -> ../../devices/pci0000:00/0000:00:1c.0/0000:09:00.3/net/eth3
The bus ID is included in the path of the link target—it is the
following string: ../../devices/pciBUS
ID/net/eth0
Create an interface map with the bus ID listed in the order the
interfaces should be used. Keep in mind that the interface from which
the node is booted using PXE must be listed first. In the following
example the default interface order has been changed to
eth0
,
eth2
,
eth1
and
eth3
.
{ "attributes" : { "network" : { "mode" : "single", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ { "pattern" : "AS 2003R", "bus_order" : [ "0000:00/0000:00:1c.0/0000:09:00.0", "0000:00/0000:00:1c.0/0000:09:00.2", "0000:00/0000:00:1c.0/0000:09:00.1", "0000:00/0000:00:1c.0/0000:09:00.3" ] } ... ], "conduit_map" : [ ... ], "networks" : { ... }, } } }
Network conduits define mappings for logical interfaces—one or more physical interfaces bonded together. Each conduit can be identified by a unique name, the “Network Mode” in this document.
. This pattern is also calledThree network modes are available:
single: Only use the first interface for all networks. VLANs will be added on top of this single interface. |
dual: Use the first interface as the admin interface and the second one for all other networks. VLANs will be added on top of the second interface. |
team: Bond the first two or more interfaces. VLANs will be added on top of the bond. |
See Section 2.1.2, “Network Modes” for detailed descriptions.
Apart from these modes a fallback mode ".*/.*/.*"
is
also pre-defined—it is applied in case no other mode matches the one
specified in the global attributes section. These modes can be adjusted
according to your needs. It is also possible to customize modes, but mode
names must be either single
, dual
, or
team
.
The mode name that is specified with mode
in the global
attributes section is deployed on all nodes in SUSE OpenStack Cloud. It is not possible
to use a different mode for a certain node. However, you can define
“sub” modes with the same name that only match the following
machines:
Machines with a certain number of physical network interfaces.
Machines with certain roles (all Compute Nodes for example).
{ "attributes" : { "network" : { "mode" : "single", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ ... ], "conduit_map" : [ { "pattern" : "single/.*/.*"1, "conduit_list" : { "intf2"2 : { "if_list" : ["1g1","1g2"]3, "team_mode" : 54 }, "intf1" : { "if_list" : ["1g1","1g2"], "team_mode" : 5 }, "intf0" : { "if_list" : ["1g1","1g2"], "team_mode" : 5 } } }, ... ], "networks" : { ... }, } } }
This line contains the pattern definition for the
MODE_NAME/NUMBER_OF_NICS/NODE_ROLE Each field in the pattern is interpreted as a Ruby regular expression (see http://www.ruby-doc.org/core-2.0/Regexp.html for a reference).
| |||||||||||||||||
The logical network interface definition. Each conduit list must contain
at least one such definition. This line defines the name of the logical
interface. This identifier must be unique and will also be referenced in
the network definition section. We recommend sticking with the
pre-defined naming scheme: | |||||||||||||||||
This line maps one or more physical interfaces to the logical interface. Each entry represents a physical interface. If more than one entry exists, the interfaces are bonded—either with the mode defined in the attribute of this conduit section. Or, if that is not present, by the globally defined attribute. The physical interfaces definition needs to fit the following pattern: [Quantifier][Speed][Order]
Valid examples are
| |||||||||||||||||
The bonding mode to be used for this logical interface. Overwrites the default set in the global attributes section for this interface. See https://www.kernel.org/doc/Documentation/networking/bonding.txt for a list of available modes. Specifying this option is optional—if not specified here, the global setting applies. |
The following example defines a team network mode for nodes with 6, 3, and an arbitrary number of network interfaces. Since the first mode that matches is applied, it is important that the specific modes (for 6 and 3 NICs) are listed before the general mode:
{ "attributes" : { "network" : { "mode" : "single", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ ... ], "conduit_map" : [ { "pattern" : "single/6/.*", "conduit_list" : { ... } }, { "pattern" : "single/3/.*", "conduit_list" : { ... } }, { "pattern" : "single/.*/.*", "conduit_list" : { ... } }, ... ], "networks" : { ... }, } } }
The following example defines network modes for Compute Nodes with four
physical interfaces, the Administration Server (role crowbar
), the
Control Node, and a general mode applying to all other nodes.
{ "attributes" : { "network" : { "mode" : "team", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ ... ], "conduit_map" : [ { "pattern" : "team/4/nova-compute", "conduit_list" : { ... } }, { "pattern" : "team/.*/^crowbar$", "conduit_list" : { ... } }, { "pattern" : "team/.*/nova-controller", "conduit_list" : { ... } }, { "pattern" : "team/.*/.*", "conduit_list" : { ... } }, ... ], "networks" : { ... }, } } }
The following values for node_role
can be used:
ceilometer-polling
|
ceilometer-server
|
cinder-controller
|
cinder-volume
|
crowbar
|
database-server
|
glance-server
|
heat-server
|
horizon-server
|
keystone-server
|
manila-server
|
manila-share
|
monasca-agent
|
monasca-log-agent
|
monasca-master
|
monasca-server
|
neutron-network
|
neutron-server
|
nova-controller
|
nova-compute-*
|
rabbitmq-server
|
trove-server
|
swift-dispersion
|
swift-proxy
|
swift-ring-compute
|
swift-storage
|
The role crowbar
refers to the Administration Server.
crowbar
and Pattern Matching
As explained in Example 7.4, “Network Modes for Certain Machines”, each node has an
additional, unique role named crowbar-FULLY
QUALIFIED HOSTNAME
.
All three elements of the value of the pattern
line
are read as regular expressions. Therefore using the pattern
mode-name/.*/crowbar
will
match all nodes in your installation. crowbar
is
considered a substring and therefore will also match all strings
crowbar-FULLY QUALIFIED
HOSTNAME
. As a consequence, all subsequent map
definitions will be ignored. To make sure this does not happen, you must
use the proper regular expression ^crowbar$
:
mode-name/.*/^crowbar$
.
Apart from the roles listed under Example 7.3, “Network Modes for Certain Roles”, each
node in SUSE OpenStack Cloud has a unique role, which lets you create modes matching
exactly one node. Each node can be addressed by its unique role name in
the pattern
entry of the
conduit_map
.
The role name depends on the fully qualified host name (FQHN) of the
respective machine. The role is named after the scheme
crowbar-FULLY QUALIFIED
HOSTNAME
where colons are replaced with dashes,
and periods are replaced with underscores. The FQHN depends on whether the
respective node was booted via PXE or not.
To determine the host name of a node, log in to the Crowbar Web interface and got to
› . Click the respective node name to get detailed data for the node. The FQHN is listed first under .
The FULLY QUALIFIED HOSTNAME for nodes
booted via PXE is composed of the following: a prefix 'd', the MAC
address of the network interface used to boot the node via PXE, and the
domain name as configured on the Administration Server. A machine with the fully
qualified host name
d1a-12-05-1e-35-49.cloud.example.com
would get the
following role name:
crowbar-d1a-12-05-1e-35-49_cloud_example_com
The fully qualified hostnames of the Administration Server and all nodes added manually (as described in Section 11.3, “Converting Existing SUSE Linux Enterprise Server 12 SP3 Machines Into SUSE OpenStack Cloud Nodes”) are defined by the system administrator. They typically have the form hostname+domain, for example admin.cloud.example.com, which would result in the following role name:
crowbar-admin_cloud_example_com
Network mode definitions for certain machines must be listed first in the conduit map. This prevents other, general rules which would also map from being applied.
{ "attributes" : { "network" : { "mode" : "dual", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ ... ], "conduit_map" : [ { "pattern" : "dual/.*/crowbar-d1a-12-05-1e-35-49_cloud_example_com", "conduit_list" : { ... } }, ... ], "networks" : { ... }, } } }
The network definitions contain IP address assignments, the bridge and VLAN setup, and settings for the router preference. Each network is also assigned to a logical interface defined in the network conduit section. In the following the network definition is explained using the example of the admin network definition:
{ "attributes" : { "network" : { "mode" : "single", "start_up_delay" : 30, "teaming" : { "mode": 5 }, "enable_tx_offloading" : true, "enable_rx_offloading" : true, "interface_map" : [ ... ], "conduit_map" : [ ... ], "networks" : { "admin" : { "conduit" : "intf0"1, "add_bridge" : false2, "use_vlan" : false3, "vlan" : 1004, "router_pref" : 105, "subnet" : "192.168.124.0"6, "netmask" : "255.255.255.0", "router" : "192.168.124.1", "broadcast" : "192.168.124.255", "ranges" : { "admin" : { "start" : "192.168.124.10", "end" : "192.168.124.11" }, "switch" : { "start" : "192.168.124.241", "end" : "192.168.124.250" }, "dhcp" : { "start" : "192.168.124.21", "end" : "192.168.124.80" }, "host" : { "start" : "192.168.124.81", "end" : "192.168.124.160" } } }, "nova_floating": { "add_ovs_bridge": false7, "bridge_name": "br-public"8, .... } ... }, } } }
Logical interface assignment. The interface must be defined in the network conduit section and must be part of the active network mode. | |
Bridge setup. Do not touch. Should be | |
Create a VLAN for this network. Changing this setting is not recommended. | |
ID of the VLAN. Change this to the VLAN ID you intend to use for the
specific network, if required. This setting can also be changed using the
YaST Crowbar interface. The VLAN ID for the
| |
Router preference, used to set the default route. On nodes hosting
multiple networks the router with the lowest
| |
Network address assignments. These values can also be changed by using the YaST Crowbar interface. | |
Openvswitch virtual switch setup. This attribute is maintained by Crowbar on a per-node level and should not be changed manually. | |
Name of the openvswitch virtual switch. This attribute is maintained by Crowbar on a per-node level and should not be changed manually. |
As of SUSE OpenStack Cloud Crowbar 8, using a VLAN for the admin network is only supported on a native/untagged VLAN. If you need VLAN support for the admin network, it must be handled at switch level.
When changing the network configuration with YaST or by editing
/etc/crowbar/network.json
, you can define VLAN
settings for each network. For the networks nova-fixed
and nova-floating
, however, special rules apply:
nova-fixed: The setting will be ignored. However, VLANs will automatically be used if deploying Neutron with VLAN support (using the plugins linuxbridge, openvswitch plus VLAN, or cisco plus VLAN). In this case, you need to specify a correct for this network.
nova-floating: When using a VLAN for
nova-floating
(which is the default), the and settings for
and default to
the same.
You have the option of separating public and floating networks with a
custom configuration. Configure your own separate floating network (not as
a subnet of the public network), and give the floating network its own
router. For example, define nova-floating
as part of an
external network with a custom bridge-name
. When you
are using different networks and OpenVSwitch is configured, the
pre-defined bridge-name
won't work.
By default, external networks cannot be reached from nodes in the SUSE OpenStack Cloud.
To access external services such as a SUSE Manager server, an SMT server, or
a SAN, you need to make the external network(s) known to SUSE OpenStack Cloud. Do so by
adding a network definition for each external network to
/etc/crowbar/network.json
. Refer to
Section 7.5, “Custom Network Configuration” for setup
instructions.
"external" : { "add_bridge" : false, "vlan" : XXX, "ranges" : { "host" : { "start" : "192.168.150.1", "end" : "192.168.150.254" } }, "broadcast" : "192.168.150.255", "netmask" : "255.255.255.0", "conduit" : "intf1", "subnet" : "192.168.150.0", "use_vlan" : true }
Replace the value XXX for the VLAN by a value not used within the SUSE OpenStack Cloud network and not used by Neutron. By default, the following VLANs are already used:
VLAN ID |
Used by |
---|---|
100 |
BMC VLAN (bmc_vlan) |
200 |
Storage Network |
300 |
Public Network (nova-floating, public) |
400 |
Software-defined network (os_sdn) |
500 |
Private Network (nova-fixed) |
501 - 2500 |
Neutron (value of nova-fixed plus 2000) |
For custom setups, the public and floating networks can be separated.
Configure your own separate floating network (not as a subnet of the public
network), and give the floating network its own router. For example, define
nova-floating
as part of an external network with a
custom bridge-name
. When you are using different
networks and OpenVSwitch is configured, the pre-defined
bridge-name
won't work.
If you need to adjust the Maximum Transmission Unit (MTU) for the Admin
and/or Storage Network, adjust
/etc/crowbar/network.json
as shown below. You can also
enable jumbo frames this way by setting the MTU to 9000. The following
example enables jumbo frames for both, the storage and the admin network by
setting "mtu": 9000
.
"admin": { "add_bridge": false, "broadcast": "192.168.124.255", "conduit": "intf0", "mtu": 9000, "netmask": "255.255.255.0", "ranges": { "admin": { "end": "192.168.124.11", "start": "192.168.124.10" }, "dhcp": { "end": "192.168.124.80", "start": "192.168.124.21" }, "host": { "end": "192.168.124.160", "start": "192.168.124.81" }, "switch": { "end": "192.168.124.250", "start": "192.168.124.241" } }, "router": "192.168.124.1", "router_pref": 10, "subnet": "192.168.124.0", "use_vlan": false, "vlan": 100 }, "storage": { "add_bridge": false, "broadcast": "192.168.125.255", "conduit": "intf1", "mtu": 9000, "netmask": "255.255.255.0", "ranges": { "host": { "end": "192.168.125.239", "start": "192.168.125.10" } }, "subnet": "192.168.125.0", "use_vlan": true, "vlan": 200 },
After you have completed the SUSE OpenStack Cloud Crowbar installation, you cannot change the network setup, and you cannot change the MTU size.
SUSE OpenStack Cloud includes a new script, network-json-resolve
,
which matches the physical and logical names of network interfaces, and
prints them to stdout. Use this to verify that you are using the correct
interface names in network.json
. Note that it will
only work if OpenStack nodes have been deployed. The following command
prints a help menu:
sudo /opt/dell/bin/network-json-resolve -h
network-json-resolve
reads your deployed
network.json
file. To use a different
network.json
file, specify its full path with the
--network-json
option. The following example shows how
to use a different network.json
file, and prints the
interface mappings of a single node:
sudo /opt/dell/bin/network-json-resolve --network-json /opt/configs/network.json aliases compute1
eth0: 0g1, 1g1
eth1: 0g1, 1g1
You may query the mappings of a specific network interface:
sudo /opt/dell/bin/network-json-resolve aliases compute1 eth0
eth0: 0g1, 1g1
Print the bus ID order on a node. This returns no bus order
defined for node
if you did not
configure any bus ID mappings:
sudo /opt/dell/bin/network-json-resolve bus_order compute1
Print the defined conduit map for the node:
sudo /opt/dell/bin/network-json-resolve conduit_map compute1
bastion: ?1g1
intf0: ?1g1
intf1: ?1g1
intf2: ?1g1
Resolve conduits to the standard interface names:
sudo /opt/dell/bin/network-json-resolve conduits compute1
bastion:
intf0: eth0
intf1: eth0
intf2: eth0
Resolve the configured networks on a node to the standard interface names:
sudo /opt/dell/bin/network-json-resolve networks compute1
bastion:
bmc_vlan: eth0
nova_fixed: eth0
nova_floating: eth0
os_sdn: eth0
public: eth0
storage: eth0
Resolve the specified network to the standard interface name(s):
sudo /opt/dell/bin/network-json-resolve networks compute1 public
public: eth0
Resolve a network.json
-style interface to its standard
interface name(s):
sudo /opt/dell/bin/network-json-resolve resolve compute1 1g1
eth0
The last step in configuring the Administration Server is starting Crowbar.
Before starting the SUSE OpenStack Cloud Crowbar installation to finish the configuration of the Administration Server, make sure to double-check the following items.
Make sure the network configuration is correct. Run Chapter 7, Crowbar Setup for further instructions.
› to review/change the configuration. SeeIf you are planning to make SUSE OpenStack Cloud highly available, whether upon the initial setup or later, set up the network in the team mode. Such a setup requires at least two network cards for each node.
Make sure hostname
-f
returns a
fully qualified host name. See
Chapter 6, Service Configuration: Administration Server Network Configuration for further instructions.
Make sure all update and product repositories are available. See Chapter 5, Software Repository Setup for further instructions.
Make sure the operating system and SUSE OpenStack Cloud Crowbar are up-to-date and
have the latest patches installed. Run zypper patch
to install them.
To use the Web interface for the SUSE OpenStack Cloud Crowbar installation you need network access to the Administration Server via a second network interface. As the network will be reconfigured during the SUSE OpenStack Cloud Crowbar installation, make sure to either have a bastion network or an external gateway configured. (For details on bastion networks, see Section 7.3.1, “Setting Up a Bastion Network”.)
Now everything is in place to finally set up Crowbar and install the Administration Server. Crowbar requires a MariaDB database—you can either create one on the Administration Server or use an existing MariaDB database on a remote server.
Start Crowbar:
sudo systemctl start crowbar-init
Create a new database on the Administration Server. By default the credentials
crowbar
/crowbar
are used:
crowbarctl database create
To use a different user name and password, run the following command instead:
crowbarctl database create \ --db_username=USERNAME --db_password=PASSWORD
Run crowbarctl database help create
for help and more
information.
Start Crowbar:
sudo systemctl start crowbar-init
Make sure a user account that can be used for the Crowbar database exists on the remote MariaDB database. If not, create such an account.
Test the database connection using the credentials from the previous step:
crowbarctl database test --db-username=USERNAME \ --db-password=PASSWORD --database=DBNAME \ --host=IP_or_FQDN --port=PORT
You need to be able to successfully connect to the database before you
can proceed. Run crowbarctl database help test
for
help and more information.
To connect to the database, use the following command:
crowbarctl database connect --db-username=USERNAME \ --db-password=PASSWORD --database=DBNAME \ --host=IP_or_FQDN --port=PORT
Run crowbarctl database help connect
for help and more
information.
After the database is successfully created and you can connect to it, access the Web interface from a Web browser, using the following address:
http://ADDRESS
Replace ADDRESS either with the IP address of the
second network interface or its associated host name. Logging in to the
Web interface requires the credentials you configured with YaST Crowbar (see Section 7.1, “). If you have not changed the
defaults, user name and password are both ”crowbar
. Refer
to Chapter 10, The Crowbar Web Interface for details.
The Web interface shows the SUSE OpenStack Cloud installation wizard. Click to begin. The installation progress is shown in the Web interface:
If the installation has successfully finished, you will be redirected to the Crowbar Dashboard:
From here you can start allocating nodes and then deploy the OpenStack services. Refer to Part III, “Setting Up OpenStack Nodes and Services” for more information.
In large deployments with many nodes, there are always some nodes that are
in a fail or unknown state. New barclamps cannot be applied to them and
values cannot be updated in some barclamps that are already deployed. This
happens because Crowbar will refuse to apply a barclamp to a list of nodes if
they are not all in ready
state.
To avoid having to manually take out nodes that are not
ready
, there is a feature called skip unready
nodes
. Instead of refusing to apply the barclamp, it will skip the
nodes that it finds in any other state than ready
.
Enabling the Feature
In /opt/dell/crowbar_framework/config/crowbar.yml
, set
the option skip_unready_nodes
to true
.
default: &default
skip_unready_nodes:
enabled: false <<< change to true
Roles Affected
All Barclamp roles are affected. The default config file includes all the
roles that have been tested and found to be working. Adding roles to the
default list is not supported for the skip_unready_nodes
feature. Removing default roles is supported.
The list of currently supported roles to skip:
- bmc-nat-client - ceilometer-agent - deployer-client - dns-client - ipmi - logging-client - nova-compute-ironic - nova-compute-kvm - nova-compute-qemu - nova-compute-vmware - nova-compute-xen - ntp-client - provisioner-base - suse-manager-client - swift-storage - updater
Determining Which Nodes Were Skipped
Skipped nodes are logged to the Crowbar log
(/var/log/crowbar/production.log
) where you can search
for the text skipped until next chef run
. This will print
the log lines where nodes were skipped, the name of the node, and the
barclamp which was being applied.
tux >
grep "skipped until next chef run" /var/log/crowbar/production.log
After enabling/disabling the skip_unready_nodes
feature
or adding/removing roles, the Crowbar framework service must be restarted
(systemctl restart crowbar
) in order to use the updated
settings.
When a barclamp is applied, all nodes will run chef-client
. Sometimes it is
not necessary to run chef for each node as attributes or roles may have not
changed for some of the nodes.
The skip unchanged nodes
feature allows nodes to be
skipped if their attributes or roles have not changed. This can help speed
up applying a barclamp, especially in large deployments and with roles that
are usually applied to a lot of nodes, such as Nova.
Enabling the Feature
In /opt/dell/crowbar_framework/config/crowbar.yml
, set
the option skip_unchanged_nodes
to
true
.
default: &default
skip_unchanged_nodes:
enabled: false <<< change to true
When a service configuration has changed, Chef forces this service to restart. Sometimes it is useful to have manual control of these restarts. This feature supports avoiding automatic restart of services and including them in a pending restart list. Disabling restarts is enabled/disabled by barclamp and cannot be done on a service level. In other words, enabling this feature on the Cinder barclamp will disable automatic restarts for all Cinder-* services.
Two steps are necessary to activate this feature:
Enable disallow_restart
in the Crowbar configuration file
Set the disable_restart
flag for a specific barclamp
using crowbar-client
or API
Enabling the Feature
In /opt/dell/crowbar_framework/config/crowbar.yml
, set
the option disallow_restart
to true
.
default: &default
disallow_restart:
enabled: false <<< change to true
The disable_restart
flag can be set with the Crowbar
client or with the API.
Set Flag with Crowbar Client
The crowbar client options for this feature are accessed with the
crowbarctl services
command, and only work for
OpenStack services. The options are:
The parameters are the barclamp name and the flag value
(true
to disable automatic restart and
false
to enable automatic restart). The command is
crowbarctl services disable_restart
BARCLAMP <true|false>
. For
example, to disable restart of the Keystone barclamp, enter the
command crowbarctl services disable_restart keystone
true
.
Used to check the disable_restart
flag for each
barclamp
crowbarctl services restart_flags { "nova": true, "keystone": true, "database": true }
Displays the list of pending restarts. The pending
restart
flag indicates that a service tried to
restart due to the Chef run but it did not due to the automatic
restart being disabled. It also indicates that the service might have
a new configuration and it will not be applied until it is manually
restarted.
In the following example, the pacemaker_service
attribute indicates whether this service is managed by Pacemaker
(usually in an HA environment) or it is a standalone service managed
by systemd
(usually in non-HA environments). More
on Pacemaker at Section 12.2, “Deploying Pacemaker (Optional, HA Setup Only)”. The
service to restart will be apache2
not managed by
Pacemaker.
crowbarctl services list_restarts { "NODE_IP": { "alias": "controller1", "keystone": { "apache2": { "pacemaker_service": false, "timestamp": "2017-11-22 11:17:49 UTC" } } } }
Removes the flag on a specific node for a specific service. It can
be executed when the service has restarted manually. The command is
crowbarctl services clear_restart
NODE
SERVICE
. For example, crowbarctl
services clear_restart NODE_IP
apache2
Using the API to List, Get Status, Set and Clear Flags
/restart_management/configuration
Lists the barclamps and the status of service reboots disallowed
Sets the disallow_restart flag for a barclamp
/restart_management/restarts
Lists all of the services needing restarts
Clears the restart flag for the given service in the given node
Sometimes a change in a proposal requires services to be restarted on all
implicated nodes. This could be a problem in a production environment where
interrupting a service completely is not an option, and where manual restart
control is needed. With the service disable_restart
feature in crowbarctl
, you can set a flag for certain
proposals to prevent the automatic restart.
Enabling the Feature
In /opt/dell/crowbar_framework/config/crowbar.yml
, set
the option disallow_restart
to true
.
default: &default
disallow_restart:
enabled: false <<< change to true
Restart Crowbar
Enable the disable_restart
flag in the affected
cookbook.
crowbarctl services disable_restart COOKBOOK true
For example, to disable Keystone and RabbitMQ restarts:
crowbarctl services disable_restart keystone true crowbarctl services disable_restart rabbitmq true
To check the proposals that have disable_restart
enabled:
crowbarctl services restart_flags { "rabbitmq": true, "keystone": true, }
Check Pending Restarts
When a proposal is applied with the disable_restart
flag
enabled, the implicated nodes will not restart. They will be listed as
pending restarts. To check this list, run the command crowbarctl
services list_restarts
.
In the following example, disable_restart
is enabled for
RabbitMQ. After applying rabbitmq, list_restarts
will
show the affected nodes, the proposal, and the services to restart.
crowbarctl services list_restarts { "d52-54-77-77-01-01.vo5.cloud.suse.de": { "alias": "controller1", "rabbitmq": { "rabbitmq-server": { "pacemaker_service": false, "timestamp": "2018-03-07 15:30:30 UTC" } } } }
After a manual restart of the service, the flags should be cleaned. The following command will clean the flag for a specific node, for a specific proposal or all proposals, and for a specific service.
crowbarctl services clear_restart NODE [COOKBOOK [SERVICE]]
For example, to clean the RabbitMQ flag from the
controller1
node, run the command:
crowbarctl service clear_restart controller1 rabbitmq
To clean the controller1
node, run the command:
crowbarctl service clear_restart controller1
The Crowbar Web interface runs on the Administration Server. It provides an
overview of the most important deployment details in your cloud. This includes a
view of the nodes and which roles are deployed on which nodes, and the
barclamp proposals that can be edited and deployed. In addition, the
Crowbar Web interface shows details about the networks and switches in your
cloud. It also provides graphical access to tools for managing
your repositories, backing up or restoring the Administration Server, exporting the
Chef configuration, or generating a supportconfig
TAR
archive with the most important log files.
The OpenStack nodes represent the actual cloud infrastructure. Node installation and service deployment is done automatically from the Administration Server. Before deploying the OpenStack services, SUSE Linux Enterprise Server 12 SP3 will be installed on all Control Nodes and Storage Nodes.
After the nodes are installed and configured you can start deploying the OpenStack components to finalize the installation. The components need to be deployed in a given order, because they depend on one another. The Pacemaker component for an HA setup is the only exception from this rule—it can be …
To limit users' access rights (or to define more fine-grained access rights), you can use Role Based Access Control (RBAC, only available with Keystone v3). In the example below, we will create a new role (ProjectAdmin). It allows users with this role to add and remove other users to the Member role…
Typically, each OpenStack component comes with a configuration file, for
example: /etc/nova/nova.conf
.
These configuration files can still be used. However, to configure an
OpenStack component and its different components and roles, it is now
preferred to add custom configuration file snippets to a
SERVICE.conf.d/
directory
instead.
This chapter describes how to install SUSE CaaS Platform Heat template on SUSE OpenStack Cloud Crowbar.
The Crowbar Web interface runs on the Administration Server. It provides an
overview of the most important deployment details in your cloud. This includes a
view of the nodes and which roles are deployed on which nodes, and the
barclamp proposals that can be edited and deployed. In addition, the
Crowbar Web interface shows details about the networks and switches in your
cloud. It also provides graphical access to tools for managing
your repositories, backing up or restoring the Administration Server, exporting the
Chef configuration, or generating a supportconfig
TAR
archive with the most important log files.
You can access the Crowbar API documentation from the following static page:
http://CROWBAR_SERVER/apidoc
.
The documentation contains information about the crowbar API endpoints and its parameters, including response examples, possible errors (and their HTTP response codes), parameter validations, and required headers.
The Crowbar Web interface uses the HTTP protocol and port
80
.
On any machine, start a Web browser and make sure that JavaScript and cookies are enabled.
As URL, enter the IP address of the Administration Server, for example:
http://192.168.124.10/
Log in as user
crowbar
. If you have not changed
the password, it is crowbar
by default.
After logging in to the Crowbar Web interface, select
› .Select the Crowbar
barclamp entry and
the proposal.
In the
section, click to edit the configuration file.Search for the following entry:
"crowbar": { "password": "crowbar"
Change the password.
Confirm your change by clicking
and .After logging in to Crowbar, you will see a navigation bar at the top-level row. Its menus and the respective views are described in the following sections.
This is the default view after logging in to the Crowbar Web interface. The Dashboard shows the groups (which you can create to arrange nodes according to their purpose), which nodes belong to each group, and which state the nodes and groups are in. In addition, the total number of nodes is displayed in the top-level row.
The color of the dot in front of each node or group indicates the status. If the dot for a group shows more than one color, hover the mouse pointer over the dot to view the total number of nodes and the statuses they are in.
Gray means the node is being discovered by the Administration Server, or that there is no up-to-date information about a deployed node. If the status is shown for a node longer than expected, check if the chef-client is still running on the node.
Yellow means the node has been successfully
Discovered
. As long as the node has not been
allocated the dot will flash. A solid (non-flashing) yellow dot
indicates that the node has been allocated, but installation has not
yet started.
Flashing from yellow to green means the node has been allocated and is currently being installed.
Solid green means the node is in status Ready
.
Red means the node is in status Problem
.
During the initial state of the setup, the Dashboard only shows
one group called sw_unknown
into which the
Administration Server is automatically sorted. Initially, all nodes (except
the Administration Server) are listed with their MAC address as a name.
However, we recommend creating an alias for each node. This makes
it easier to identify the node in the admin network and on the
Dashboard. For details on how to create groups, how to assign nodes
to a group, and how to create node aliases, see Section 11.2, “Node Installation”.
This screen allows you to edit multiple nodes at once instead of editing them individually. It lists all nodes, including OpenStack Cloud network), , , (the operating system that is going to be installed on the node), (if available), and allocation status. You can toggle the list view between or nodes.
(in form of the MAC address), configuration, (used within the admin network), (name used outside of the SUSEFor details on how to fill in the data for all nodes and how to start the installation process, see Section 11.2, “Node Installation”.
This menu entry only appears if your cloud contains a High Availability setup. The overview shows all clusters in your setup, including the
that are members of the respective cluster and the assigned to the cluster. It also shows if a cluster contains and which roles are assigned to the remote nodes.This overview shows which roles have been deployed on which node(s). The roles are grouped according to the service to which they belong. You cannot edit anything here. To change role deployment, you need to edit and redeploy the appropriate barclamps as described in Chapter 12, Deploying the OpenStack Services.
This screen shows a list of all available barclamp proposals, including their Section 10.3, “Deploying Barclamp Proposals”.
, , and a short . From here, you can individual barclamp proposals as described inThis screen only shows the barclamps that are included with the core Crowbar framework. They contain general recipes for setting up and configuring all nodes. From here, you can
individual barclamp proposals.This screen only shows the barclamps that are dedicated to OpenStack service deployment and configuration. From here, you can individual barclamp proposals.
If barclamps are applied to one or more nodes that are not yet available for deployment (for example, because they are rebooting or have not been fully installed yet), the proposals will be put in a queue. This screen shows the proposals that are
or .The supportconfig
TAR
archive. The supportconfig
archive contains system
information such as the current kernel version being used, the hardware, RPM
database, partitions, and the most important log files for analysis of any
problems. To access the export options, click . After the export has been successfully finished, the
screen will show any files that are
available for download.
This screen shows an overview of the mandatory, recommended, and optional repositories for all architectures of SUSE OpenStack Cloud Crowbar. On each reload of the screen the Crowbar Web interface checks the availability and status of the repositories. If a mandatory repository is not present, it is marked red in the screen. Any repositories marked green are usable and available to each node in the cloud. Usually, the available repositories are also shown as in the rightmost column. This means that the managed nodes will automatically be configured to use this repository. If you disable the check box for a repository, managed nodes will not use that repository.
You cannot edit any repositories in this screen. If you need additional, third-party
repositories, or want to modify the repository metadata, edit
/etc/crowbar/repos.yml
. Find an example of a repository
definition below:
suse-12.3: x86_64: Custom-Repo-12.3: url: 'http://example.com/12-SP3:/x86_64/custom-repo/' ask_on_error: true # sets the ask_on_error flag in # the autoyast profile for that repo priority: 99 # sets the repo priority for zypper
Alternatively, use the YaST Crowbar module to add or edit repositories as described in Section 7.4, “. ”
This screen allows you to run
swift-dispersion-report
on the node or nodes
to which it has been deployed. Use this tool to measure the
overall health of the swift cluster. For details, see http://docs.openstack.org/liberty/config-reference/content/object-storage-dispersion.html.
This screen is for creating and downloading a backup of the Administration Server. You can also restore from a backup or upload a backup image from your local file system.
SUSE OpenStack Cloud Crowbar can communicate with a Cisco UCS Manager instance via its XML-based API server to perform the following functions:
Instantiate UCS service profiles for Compute Nodes and Storage Nodes from predefined UCS service profile templates.
Reboot, start, and stop nodes.
The following prerequisites need to be fulfilled on the Cisco UCS side:
Templates for Compute Nodes and Storage Nodes need to be created. These
service profile templates will be used for preparing systems as
SUSE OpenStack Cloud nodes. Minimum requirements are a processor supporting
AMD-V or Intel-VT, 8 GB RAM, one network interface and at least 20 GB
of storage (more for Storage Nodes). The templates must be named
suse-cloud-compute
and suse-cloud-storage
.
A user account with administrative permissions needs to be created for communicating with SUSE OpenStack Cloud. The account needs to have access to the service profile templates listed above. It also need permission to create service profiles and associate them with physical hardware.
To initially connect to the Cisco UCS Manager, provide the login
credentials of the user account mentioned above. The http://UCSMANAGERHOST/nuova
. Click
to connect. When connected, you will see a
list of servers and associated actions. Applying an action with the
button can take up to several minutes.
From this screen you can access HTML and PDF versions of the SUSE OpenStack Cloud Crowbar manuals that are installed on the Administration Server.
Barclamps are a set of recipes, templates, and installation instructions. They are used to automatically install OpenStack components on the nodes. Each barclamp is configured via a so-called proposal. A proposal contains the configuration of the service(s) associated with the barclamp and a list of machines onto which to deploy the barclamp.
Most barclamps consist of two sections:
For changing the barclamp's configuration, either by editing the respective Web forms (
view) or by switching to the view, which exposes all configuration options for the barclamp. In the view, you directly edit the configuration file.Before you switch to
view or back again to view, your changes. Otherwise they will be lost.Lets you choose onto which nodes to deploy the barclamp. On the left-hand side, you see a list of
. The right-hand side shows a list of roles that belong to the barclamp.Assign the nodes to the roles that should be deployed on that node. Some barclamps contain roles that can also be deployed to a cluster. If you have deployed the Pacemaker barclamp, the “normal” nodes and Pacemaker remote nodes. See Section 2.6.3, “High Availability of the Compute Node(s)” for the basic details.
section additionally lists and in this case. The latter are clusters that contain bothClusters (or clusters with remote nodes) cannot be assigned to roles that need to be deployed on individual nodes. If you try to do so, the Crowbar Web interface shows an error message.
If you assign a cluster with remote nodes to a role that can only be applied to “normal” (Corosync) nodes, the role will only be applied to the Corosync nodes of that cluster. The role will not be applied to the remote nodes of the same cluster.
The following procedure shows how to generally edit, create and deploy barclamp proposals. For the description and deployment of the individual barclamps, see Chapter 12, Deploying the OpenStack Services.
Log in to the Crowbar Web interface.
Click
and select . Alternatively, filter for categories by selecting either or .To create a new proposal or edit an existing one, click
or next to the appropriate barclamp.Change the configuration in the
section:Change the available options via the Web form.
To edit the configuration file directly, first save changes made in the Web form. Click
to edit the configuration in the editor view.After you have finished,
your changes. (They are not applied yet).Assign nodes to a role in the
section of the barclamp. By default, one or more nodes are automatically pre-selected for available roles.If this pre-selection does not meet your requirements, click the
icon next to the role to remove the assignment.To assign a node or cluster of your choice, select the item you want from the list of nodes or clusters on the left-hand side, then drag and drop the item onto the desired role name on the right.
Do not drop a node or cluster onto the text box—this is used to filter the list of available nodes or clusters!
To save your changes without deploying them yet, click
.Deploy the proposal by clicking
.If you deploy a proposal onto a node where a previous one is still active, the new proposal will overwrite the old one.
Deploying a proposal might take some time (up to several minutes). Always wait until you see the message “Successfully applied the proposal” before proceeding to the next proposal.
A proposal that has not been deployed yet can be deleted in the Section 10.3.3, “Deleting a Proposal That Already Has Been Deployed”.
view by clicking . To delete a proposal that has already been deployed, seeA deployment failure of a barclamp may leave your node in an inconsistent state. If deployment of a barclamp fails:
Fix the reason that has caused the failure.
Re-deploy the barclamp.
For help, see the respective troubleshooting section at Q & A 2, “OpenStack Node Deployment”.
To delete a proposal that has already been deployed, you first need to
it.Log in to the Crowbar Web interface.
Click
› .Click
to open the editing view.Click
and confirm your choice in the following pop-up.Deactivating a proposal removes the chef role from the nodes, so the routine that installed and set up the services is not executed anymore.
Click
to confirm your choice in the following pop-up.This removes the barclamp configuration data from the server.
However, deactivating and deleting a barclamp that already had been deployed does not remove packages installed when the barclamp was deployed. Nor does it stop any services that were started during the barclamp deployment. On the affected node, proceed as follows to undo the deployment:
Stop the respective services:
root #
systemctl stop service
Disable the respective services:
root #
systemctl disable service
Uninstalling the packages should not be necessary.
When a proposal is applied to one or more nodes that are not yet available for deployment (for example, because they are rebooting or have not been yet fully installed), the proposal will be put in a queue. A message like
Successfully queued the proposal until the following become ready: d52-54-00-6c-25-44
will be shown when having applied the proposal. A new button
will also become available. Use it to cancel the deployment of the proposal by removing it from the queue.The OpenStack nodes represent the actual cloud infrastructure. Node installation and service deployment is done automatically from the Administration Server. Before deploying the OpenStack services, SUSE Linux Enterprise Server 12 SP3 will be installed on all Control Nodes and Storage Nodes.
To prepare the installation, each node needs to be booted using PXE, which
is provided by the tftp
server
from the Administration Server. Afterward you can allocate the nodes and trigger
the operating system installation.
Make a note of the MAC address and the purpose of each node (for example, controller, block storage, object storage, compute). This will make deploying the OpenStack components a lot easier and less error-prone. It also enables you to assign meaningful names (aliases) to the nodes, which are otherwise listed with the MAC address by default.
Make sure booting using PXE (booting from the network) is enabled and configured as the primary boot-option for each node. The nodes will boot twice from the network during the allocation and installation phase. Booting from the first hard disk needs to be configured as the second boot option.
All nodes are installed using AutoYaST with the same configuration
located at
/opt/dell/chef/cookbooks/provisioner/templates/default/autoyast.xml.erb
.
If this configuration does not match your needs (for example if you
need special third party drivers) you need to make adjustments to this
file. See the https://documentation.suse.com/sles/12-SP5/single-html/SLES-autoyast/ for details.
If you change the AutoYaST configuration file, you need to re-upload
it to Chef using the following command:
knife cookbook upload -o /opt/dell/chef/cookbooks/ provisioner
root
Login
By default, the root
account on the nodes has no password
assigned, so a direct root
login is not possible. Logging in
on the nodes as root
is only possible via SSH public keys
(for example, from the Administration Server).
If you want to allow direct root
login, you can set a
password via the Crowbar Provisioner barclamp before deploying the
nodes. That password will be used for the root
account on all
OpenStack nodes. Using this method after the nodes are deployed is
not possible. In that case you would need to log in to each node via
SSH from the Administration Server and change the password manually with
passwd
.
root
Password for the OpenStack Nodes #
Create an md5-hashed root
-password, for example by using
openssl passwd
-1
.
Open a browser and point it to the Crowbar Web interface on the
Administration Server, for example http://192.168.124.10
. Log
in as user crowbar
. The
password is crowbar
by default, if you have not
changed it during the installation.
Open the barclamp menu by clicking
› . Click the barclamp entry and the proposal.Click
in the section to edit the configuration file.Add the following line to the end of the file before the last closing curly bracket:
, "root_password_hash": "HASHED_PASSWORD"
replacing "HASHED_PASSWORD" with the password you generated in the first step.
Click
.To install a node, you need to boot it first using PXE. It will be booted with an image that enables the Administration Server to discover the node and make it available for installation. When you have allocated the node, it will boot using PXE again and the automatic installation will start.
Boot all nodes that you want to deploy using PXE. The nodes will boot into the SLEShammer image, which performs the initial hardware discovery.
Booting many nodes at the same time using PXE will cause heavy load on the TFTP server, because all nodes will request the boot image at the same time. We recommend booting the nodes at different intervals.
Open a browser and point it to the Crowbar Web interface on the Administration Server,
for example http://192.168.124.10/
. Log in as user
crowbar
. The password is
crowbar
by default, if you have not changed it.
Click
› to open the .
Each node that has successfully booted will be listed as being in state
Discovered
, indicated by a yellow bullet. The nodes
will be listed with their MAC address as a name. Wait until all nodes
are listed as Discovered
before proceeding. If a node does not report as Discovered
, it
may need to be rebooted manually.
Although this step is optional, we recommend properly grouping your nodes at this stage, since it lets you clearly arrange all nodes. Grouping the nodes by role would be one option, for example control, compute and object storage (Swift).
Enter the name of a new group into the
text box and click .Drag and drop a node onto the title of the newly created group. Repeat this step for each node you want to put into the group.
To allocate all nodes, click
› . To allocate a single node, click the name of a node, then click .Deploying many nodes in bulk mode will cause heavy load on the Administration Server. The subsequent concurrent Chef client runs triggered by the nodes will require a lot of RAM on the Administration Server.
Therefore it is recommended to limit the number of concurrent “Allocations” in bulk mode. The maximum number depends on the amount of RAM on the Administration Server—limiting concurrent deployments to five up to ten is recommended.
In single node editing mode, you can also specify the
ext4
for all nodes. We recommended using the default.
Provide a meaningful
, , and a for each node, and then check the box. You can also specify the for the node. This optional setting is used to make reasonable proposals for the barclamps.By default the
is set to .
Providing an alias name will change the default node names (MAC
address) to the name you provided, making it easier to identify the
node. Furthermore, this alias will also be used as a DNS
CNAME
for the node in the admin network. As a
result, you can access the node via this alias when, for example,
logging in via SSH.
A node's OpenStack Dashboard or some APIs
(keystone-server
,
glance-server
,
cinder-controller
,
neutron-server
,
nova-controller
, and
swift-proxy
) can be accessed
from outside the SUSE OpenStack Cloud network. To be able to access them by
name, these names need to be resolved by a name server placed outside
of the SUSE OpenStack Cloud network. If you have created DNS entries for nodes,
specify the name in the field.
The OpenStack Cloud
network. However, if you create an SSL certificate for a node that has
a public name, this name must be added as an
AlternativeName
to the certificate. See Section 2.3, “SSL Encryption” for more information.
When you have filled in the data for all nodes, click
. The nodes will reboot and commence the AutoYaST-based SUSE Linux Enterprise Server installation (or installation of other target platforms, if selected) via a second boot using PXE. Click › to return to the .
Nodes that are being installed are listed with the status
Installing
(yellow/green bullet). When the
installation of a node has finished, it is listed as being
Ready
, indicated by a green bullet. Wait until all
nodes are listed as Ready
before proceeding.
SUSE OpenStack Cloud allows adding existing machines installed with SUSE Linux Enterprise Server 12 SP3 to the pool of nodes. This enables you to use spare machines for SUSE OpenStack Cloud, and offers an alternative way of provisioning and installing nodes (via SUSE Manager for example). The machine must run SUSE Linux Enterprise Server 12 SP3.
The machine also needs to be on the same network as the Administration Server, because it needs to communicate with this server. Since the Administration Server provides a DHCP server, we recommend configuring this machine to get its network assignments from DHCP. If it has a static IP address, make sure it is not already used in the admin network. Check the list of used IP addresses with the YaST Crowbar module as described in Section 7.2, “. ”
Proceed as follows to convert an existing SUSE Linux Enterprise Server 12 SP3 machine into a SUSE OpenStack Cloud node:
Download the crowbar_register
script from the
Administration Server at
http://192.168.124.10:8091/suse-12.3/x86_64/crowbar_register
.
Replace the IP address with the IP address of your Administration Server using
curl
or wget
. Note that the
download only works from within the admin network.
Make the crowbar_register
script executable
(chmod
a+x
crowbar_register).
Run the crowbar_register
script. If you have
multiple network interfaces, the script tries to automatically detect
the one that is connected to the admin network. You may also explicitly
specify which network interface to use by using the
--interface
switch, for example
crowbar_register
--interface eth1
.
After the script has successfully run, the machine has been added to the pool of nodes in the SUSE OpenStack Cloud and can be used as any other node from the pool.
The following lists some optional configuration steps like configuring node updates, monitoring, access, and enabling SSL. You may entirely skip the following steps or perform any of them at a later stage.
To keep the operating system and the SUSE OpenStack Cloud software itself up-to-date on the nodes, you can deploy either the Updater barclamp or the SUSE Manager barclamp. The latter requires access to a SUSE Manager server. The Updater barclamp uses Zypper to install updates and patches from repositories made available on the Administration Server.
The easiest way to provide the required repositories on the Administration Server is to set up an SMT server as described in Chapter 4, Installing and Setting Up an SMT Server on the Administration Server (Optional). Alternatives to setting up an SMT server are described in Chapter 5, Software Repository Setup.
The Updater barclamp lets you deploy updates that are available on the update repositories at the moment of deployment. Each time you deploy updates with this barclamp you can choose a different set of nodes to which the updates are deployed. This lets you exactly control where and when updates are deployed.
To deploy the Updater barclamp, proceed as follows. For general instructions on how to edit barclamp proposals refer to Section 10.3, “Deploying Barclamp Proposals”.
Open a browser and point it to the Crowbar Web interface on the
Administration Server, for example http://192.168.124.10/
. Log in
as user crowbar
. The password
is crowbar
by default, if you have not changed it
during the installation.
Open the barclamp menu by clicking
› . Click the barclamp entry and to open the proposal.Configure the barclamp by the following attributes. This configuration always applies to all nodes on which the barclamp is deployed. Individual configurations for certain nodes are only supported by creating a separate proposal.
Define which Zypper subcommand to use for updating.
will install all patches applying to the system from the configured update repositories that are available. will update packages from all configured repositories (not just the update repositories) that have a higher version number than the installed packages. replaces each package installed with the version from the repository and deletes packages not available in the repositories.We recommend using
.If set to true (recommended), checks if packages are correctly signed.
If set to true (recommended), Zypper automatically accepts third party licenses.
Installs patches that require a reboot (for example Kernel or glibc
updates). Only set this option to true
when you
can safely reboot the affected nodes. Refer to
Chapter 17, SUSE OpenStack Cloud Maintenance for more information.
Installing a new Kernel and not rebooting may result in an unstable
system.
Automatically reboots the system in case a patch requiring a reboot
has been installed. Only set this option to true
when you can safely reboot the affected nodes. Refer to
Chapter 17, SUSE OpenStack Cloud Maintenance for more information.
Choose the nodes on which the Updater barclamp should be deployed in the
section by dragging them to the column.
zypper
keeps track of the packages and patches it
installs in /var/log/zypp/history
. Review that log
file on a node to find out which updates have been installed. A second
log file recording debug information on the zypper
runs can be found at /var/log/zypper.log
on each
node.
Before starting an update for a cluster node, either stop the cluster stack on that node or put the cluster into maintenance mode. If the cluster resource manager on a node is active during the software update, this can lead to unpredictable results like fencing of active nodes. For detailed instructions refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-migration-update.
To keep the operating system and the SUSE OpenStack Cloud software itself up-to-date on the nodes, you can deploy either barclamp or the Updater barclamp. The latter uses Zypper to install updates and patches from repositories made available on the Administration Server.
To enable the SUSE Manager server to manage the SUSE OpenStack Cloud nodes, you must make the respective SUSE OpenStack Cloud Crowbar 8 channels, the SUSE Linux Enterprise Server 12 SP3 channels, and the channels for extensions used with your deployment (High Availability Extension, SUSE Enterprise Storage) available via an activation key.
The
barclamp requires access to the SUSE Manager server from every node it is deployed to.To deploy the Section 10.3, “Deploying Barclamp Proposals”.
barclamp, proceed as follows. For general instructions on how to edit barclamp proposals refer to
Download the package
rhn-org-trusted-ssl-cert-VERSION-RELEASE.noarch.rpm
from
https://susemanager.example.com/pub/.
VERSION and
RELEASE may vary, ask the administrator of
the SUSE Manager for the correct values.
susemanager.example.com needs to be
replaced by the address of your SUSE Manager server. Copy the file you
downloaded to
/opt/dell/chef/cookbooks/suse-manager-client/files/default/ssl-cert.rpm
on the Administration Server. The package contains the SUSE Manager's CA SSL
Public Certificate. The certificate installation has not been
automated on purpose, because downloading the certificate manually
enables you to check it before copying it.
Re-install the barclamp by running the following command:
/opt/dell/bin/barclamp_install.rb --rpm core
Open a browser and point it to the Crowbar Web interface on the
Administration Server, for example http://192.168.124.10/
. Log in
as user crowbar
. The password
is crowbar
by default, if you have not changed it
during the installation.
Open the barclamp menu by clicking
› . Click the barclamp entry and to open the proposal.Specify the URL of the script for activation of the clients in the
field.Choose the nodes on which the SUSE Manager barclamp should be deployed in the OpenStack Cloud.
section by dragging them to the column. We recommend deploying it on all nodes in the SUSEBefore starting an update for a cluster node, either stop the cluster stack on that node or put the cluster into maintenance mode. If the cluster resource manager on a node is active during the software update, this can lead to unpredictable results like fencing of active nodes. For detailed instructions refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-migration-update.
The NFS barclamp allows you to mount NFS share from a remote host on nodes in the cloud. This feature can, for example, be used to provide an image repository for Glance. Note that all nodes which are to mount an NFS share must be able to reach the NFS server. This requires manually adjusting the network configuration.
To deploy the NFS barclamp, proceed as follows. For general instructions on how to edit barclamp proposals refer to Section 10.3, “Deploying Barclamp Proposals”.
Open a browser and point it to the Crowbar Web interface on the
Administration Server, for example http://192.168.124.10/
. Log in
as user crowbar
. The password
is crowbar
by default, if you have not changed it
during the installation.
Open the barclamp menu by clicking
› . Click the barclamp entry and to open the proposal.Configure the barclamp by the following attributes. Each set of attributes is used to mount a single NFS share.
Unique name for the current configuration. This name is used in the Web interface only to distinguish between different shares.
Fully qualified host name or IP address of the NFS server.
Export name for the share on the NFS server.
Mount point on the target machine.
Mount options that will be used on the node. See man 8
mount
for general mount options and man 5
nfs
for a list of NFS-specific options. Note that the
general option nofail
(do not report errors if
device does not exist) is automatically set.
After having filled in all attributes, click
. If you want to mount more than one share, fill in the data for another NFS mount. Otherwise click to save the data, or to deploy the proposal. Note that you must always click before saving or applying the barclamp, otherwise the data that was entered will be lost.Go to the
section and drag and drop all nodes, on which the NFS shares defined above should be mounted, to the column. Click to deploy the proposal.The NFS barclamp is the only barclamp that lets you create different proposals, enabling you to mount different NFS shares on different nodes. When you have created an NFS proposal, a special
is shown in the barclamp overview of the Crowbar Web interface. Click it to either an existing proposal or a new one. New proposals must have unique names.The following chapter provides instructions on using an external Ceph cluster in SUSE OpenStack Cloud Crowbar.
External Ceph cluster are supported with SUSE Enterprise Storage 5 or higher. The version of Ceph should be compatible with the version of the Ceph client supplied with SUSE Linux Enterprise Server 12 SP3.
The external Ceph cluster needs to be connected to a separate VLAN, which is mapped to the SUSE OpenStack Cloud storage VLAN. See Section 2.1, “Network” for more information.
Ceph can be used from the KVM Compute Nodes, with Cinder, and with Glance. The following installation steps need to be executed on each node accessing Ceph:
The following steps need to be executed before the barclamps get deployed.
Log in as user root
to a machine in the Ceph cluster
and generate keyring files for Cinder users. Optionally, you can
generate keyring files for the Glance users (only needed
when using Glance with Ceph/Rados). The keyring file that will be
generated for Cinder will also be used on the Compute Nodes.
To do so, you need to specify pool names and user names for both services. The default
names are:
Glance |
Cinder | |
---|---|---|
User |
glance |
cinder |
Pool |
images |
volumes |
Make a note of user and pool names in case you do not use the default values. You will need this information later, when deploying Glance and Cinder.
If you decide to use the admin keyring file to connect the external Ceph cluster, be aware that after Crowbar discovers this admin keyring, it will create client keyring files, pools, and capabilities needed to run Glance, Cinder, or Nova integration.
If you have access to the admin keyring file and agree that automatic changes will be done to the cluster as described above, copy it together with the Ceph configuration file to the Administration Server. If you cannot access this file, create a keyring:
When you can access the admin keyring file
ceph.client.admin.keyring
, copy it together with
ceph.conf
(both files are usually located in
/etc/ceph
) to a temporary location on the
Administration Server, for example /root/tmp/
.
If you cannot access the admin keyring file create a new keyring file with the following commands. Re-run the commands for Glance, too, if needed. First create a key:
ceph auth get-or-create-key client.USERNAME mon "allow r" \ osd 'allow class-read object_prefix rbd_children, allow rwx \ pool=POOLNAME'
Replace USERNAME and POOLNAME with the respective values.
Now use the key to generate the keyring file
/etc/ceph/ceph.client.USERNAME.keyring
:
ceph-authtool \ /etc/ceph/ceph.client.USERNAME.keyring \ --create-keyring --name=client.USERNAME> \ --add-key=KEY
Replace USERNAME with the respective value.
Copy the Ceph configuration file ceph.conf
(usually located in /etc/ceph
) and the keyring
file(s) generated above to a temporary location on the Administration Server, for
example /root/tmp/
.
Log in to the Crowbar Web interface and check whether the nodes which should have access to the Ceph cluster already have an IP address from the storage network. Do so by going to the no storage network IP address.
and clicking the node name. An should be listed for . Make a note of the of each node that has
Log in to the Administration Server as user root
and run the
following command for all nodes you noted down in the previous step:
crowbar network allocate_ip "default" NODE "storage" "host" chef-client
NODE needs to be replaced by the node's name.
After executing the command in the previous step for all
affected nodes, run the command chef-client
on the
Administration Server.
Log in to each affected node as user root
. See
How can I log in to a node as root?
for instructions.
On each node, do the following:
Manually install nova, cinder (if using cinder) and/or glance (if using glance) packages with the following commands:
zypper in openstack-glance zypper in openstack-cinder zypper in openstack-nova
Copy the ceph.conf file from the Administration Server to
/etc/ceph
:
mkdir -p /etc/ceph scp root@admin:/root/tmp/ceph.conf /etc/ceph chmod 664 /etc/ceph/ceph.conf
Copy the keyring file(s) to /etc/ceph
. The
exact process depends on whether you have copied the admin keyring
file or whether you have created your own keyrings:
If you have copied the admin keyring file, run the following command on the Control Node(s) on which Cinder and Glance will be deployed, and on all KVM Compute Nodes:
scp root@admin:/root/tmp/ceph.client.admin.keyring /etc/ceph chmod 640 /etc/ceph/ceph.client.admin.keyring
If you have created you own keyrings, run the following command on the Control Node on which Cinder will be deployed, and on all KVM Compute Nodes to copy the Cinder keyring:
scp root@admin:/root/tmp/ceph.client.cinder.keyring /etc/ceph chmod 640 /etc/ceph/ceph.client.cinder.keyring
On Control Node on which Cinder will be deployed run the following command to update file ownership:
chown root.cinder /etc/ceph/ceph.client.cinder.keyring
On KVM Compute Nodes run the following command to update file ownership:
chown root.nova /etc/ceph/ceph.client.cinder.keyring
Now copy the Glance keyring to the Control Node on which Glance will be deployed:
scp root@admin:/root/tmp/ceph.client.glance.keyring /etc/ceph chmod 640 /etc/ceph/ceph.client.glance.keyring chown root.glance /etc/ceph/ceph.client.glance.keyring
The nodes can only be accessed via SSH from the Administration Server—it is not possible to connect to them from any other host in the network.
The root
account on the nodes has no
password assigned, therefore logging in to a node as
root
@node is only possible via SSH
with key authentication. By default, you can only log in with the key of
the root
of the Administration Server
(root@admin) via SSH only.
If you have added users to the Administration Server and want to
give them permission to log in to the nodes as well, you need to add
these users' public SSH keys to root
's
authorized_keys
file on all nodes. Proceed as
follows:
If they do not already exist, generate an SSH key pair with
ssh-keygen
. This key pair belongs to the user that you use to log in to the nodes. Alternatively, copy an existing public
key with ssh-copy-id
. Refer to the respective man
pages for more information.
Log in to the Crowbar Web interface on the Administration Server, for
example http://192.168.124.10/
(user name and default
password: crowbar
).
Open the barclamp menu by clicking
› . Click the barclamp entry and the proposal.Copy and paste the public SSH key of the user into the text box. If adding keys for multiple users, note that each key needs to be placed on a new line.
Click
to deploy the keys and save your changes to the proposal.To enable SSL to encrypt communication within the cloud (see Section 2.3, “SSL Encryption” for details), all nodes running encrypted services need SSL certificates. An SSL certificate is, at a minimum, required on the Control Node.
Each certificate consists
of a pair of files: the certificate file (for example,
signing_cert.pem
) and the key file (for example,
signing_key.pem
). If you use your own certificate
authority (CA) for signing, you will also need a certificate file for
the CA (for example, ca.pem
). We recommend copying the files to the /etc
directory using the
directory structure outlined below. If you use a dedicated certificate
for each service, create directories named after the services (for
example, /etc/keystone
). If you are using shared
certificates, use a directory such as /etc/cloud
.
/etc/cloud/ssl/certs/signing_cert.pem
/etc/cloud/private/signing_key.pem
/etc/cloud/ssl/certs/ca.pem
All nodes that have been allocated can be decommissioned or re-installed. Click a node's name in the
to open a screen with the node details. The following options are available:Deletes a node from the pool. If you want to re-use this node again, it needs to be reallocated and re-installed from scratch.
Triggers a reinstallation. The machine stays allocated. Any barclamps that were deployed on the machine will be re-applied after the installation.
Temporarily removes the node from the pool of nodes. After you reallocate the node it will take its former role. Useful for adding additional machines in times of high load or for decommissioning machines in times of low load.
Reboots the node.
Shuts the node down.
Forces a (non-clean) shuts down and a restart afterward. Only use if a reboot does not work.
Forces a (non-clean) node shut down. Only use if a clean shut down does not work.
When de-allocating nodes that provide essential services, the complete cloud will become unusable. If you have not disabled redundancy, you can disable single storage nodes or single compute nodes. However, disabling Control Node(s) will cause major problems. It will either “kill” certain services (for example Swift) or, at worst the complete cloud (when deallocating the Control Node hosting Neutron). You should also not disable the nodes providing swift ring and proxy services.
After the nodes are installed and configured you can start deploying the OpenStack components to finalize the installation. The components need to be deployed in a given order, because they depend on one another. The component for an HA setup is the only exception from this rule—it can be set up at any time. However, when deploying SUSE OpenStack Cloud Crowbar from scratch, we recommend deploying the proposal(s) first. Deployment for all components is done from the Crowbar Web interface through recipes, so-called “barclamps”. (See Section 12.23, “Roles and Services in SUSE OpenStack Cloud Crowbar” for a table of all roles and services, and how to start and stop them.)
The components controlling the cloud, including storage management and control components, need to be installed on the Control Node(s) (refer to Section 1.2, “The Control Node(s)” for more information). However, you may not use your Control Node(s) as a compute node or storage host for Swift. Do not install he components and on the Control Node(s). These components must be installed on dedicated Storage Nodes and Compute Nodes.
When deploying an HA setup, the Control Nodes are replaced by one or more controller clusters consisting of at least two nodes, and three are recommended. We recommend setting up three separate clusters for data, services, and networking. See Section 2.6, “High Availability” for more information on requirements and recommendations for an HA setup.
The OpenStack components need to be deployed in the following order. For general instructions on how to edit and deploy barclamps, refer to Section 10.3, “Deploying Barclamp Proposals”. Any optional components that you elect to use must be installed in their correct order.
Designate provides SUSE OpenStack Cloud Crowbar DNS as a Service (DNSaaS). It is used to
create and propagate zones and records over the network using pools of DNS
servers. Deployment defaults are in place, so not much is required to
configure Designate. Neutron needs additional settings for integration with
Designate, which are also present in the [designate]
section in Neutron configuration.
The Designate barclamp relies heavily on the DNS barclamp and expects it to be applied without any failures.
In order to deploy Designate, at least one node is necessary in the DNS barclamp that is not the admin node. The admin node is not added to the public network. So another node is needed that can be attached to the public network and appear in the designate default pool.
We recommend that DNS services are running in a cluster in highly available deployments where Designate services are running in a cluster. For example, in a typical HA deployment where the controllers are deployed in a 3-node cluster, the DNS barclamp should be applied to all the controllers, in the same manner as Designate.
Installs the Designate server packages and configures the mini-dns (mdns) service required by Designate.
Configures a Designate worker on the selected nodes. Designate uses the workers to distribute its workload.
Designate Sink
is an optional service and is not configured as part
of this barclamp.
Designate uses pool(s) over which it can distribute zones and records. Pools can have varied configuration. Any misconfiguration can lead to information leakage.
The Designate barclamp creates default Bind9 pool out of the box, which can be
modified later as needed. The default Bind9 pool configuration is created by Crowbar
on a node with designate-server
role in
/etc/designate/pools.crowbar.yaml
. You can copy
this file and edit it according to your requirements. Then provide this
configuration to Designate using the command:
ardana >
designate-manage pool update --file /etc/designate/pools.crowbar.yaml
The dns_domain
specified in Neutron configuration in [designate]
section
is the default Zone where DNS records for Neutron resources are created via
Neutron-Designate integration. If this is desired, you have to create this zone
explicitly using the following command:
ardana >
openstack zone create < email > < dns_domain >
Editing the Designate proposal:
Designate uses Bind9 backend by default. It is also possible to use PowerDNS backend in addition to, or as an alternative, to Bind9 backend. To do so PowerDNS must be manually deployed as The Designate barclamp currently does not provide any facility to automatically install and configure PowerDNS. This section outlines the steps to deploy PowerDNS backend.
If PowerDNS is already deployed, you may skip the Section 12.1.1.1, “Install PowerDNS” section and jump to the Section 12.1.1.2, “Configure Designate To Use PowerDNS Backend” section.
Follow these steps to install and configure PowerDNS on a Crowbar node. Keep in mind that PowerDNS must be deployed with MySQL backend.
We recommend that PowerDNS are running in a cluster in highly availability deployments where Designate services are running in a cluster. For example, in a typical HA deployment where the controllers are deployed in a 3-node cluster, PowerDNS should be running on all the controllers, in the same manner as Designate.
Install PowerDNS packages.
root #
zypper install pdns pdns-backend-mysql
Edit /etc/pdns/pdns.conf
and provide these options:
(See
https://doc.powerdns.com/authoritative/settings.html for a complete reference).
Set it to yes
to enable Web service Rest API.
Static Rest API access key. Use a secure random string here.
Must set to gmysql
to use MySQL backend.
Hostname (i.e. FQDN) or IP address of the MySQL server.
MySQL user which have full access to the PowerDNS database.
Password for the MySQL user.
MySQL database name for PowerDNS.
Port number where PowerDNS is listening for upcoming requests.
The group where the PowerDNS process is running under.
The user where the PowerDNS process is running under.
Must set to yes
to enable web service RestAPI.
Hostname (FQDN) or IP address of the PowerDNS web service.
List of IP addresses (IPv4 or IPv6) of the nodes that are permitted to talk to the PowerDNS web service. These must include the IP address of the Designate worker nodes.
For example:
api=yes api-key=Sfw234sDFw90z launch=gmysql gmysql-host=mysql.acme.com gmysql-user=powerdns gmysql-password=SuperSecured123 gmysql-dbname=powerdns local-port=54 setgid=pdns setuid=pdns webserver=yes webserver-address=192.168.124.83 webserver-allow-from=0.0.0.0/0,::/0
Login to MySQL from a Crowbar MySQL node and create the PowerDNS database
and the user which has full access to the PowerDNS database. Remember,
the database name, username, and password must match
gmysql-dbname
, gmysql-user
,
and gmysql-password
that were specified above
respectively.
For example:
root #
mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 20075
Server version: 10.2.29-MariaDB-log SUSE package
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> CREATE DATABASE powerdns;
Query OK, 1 row affected (0.01 sec)
MariaDB [(none)]> GRANT ALL ON powerdns.* TO 'powerdns'@'localhost' IDENTIFIED BY 'SuperSecured123';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> GRANT ALL ON powerdns.* TO 'powerdns'@'192.168.124.83' IDENTIFIED BY 'SuperSecured123';
Query OK, 0 rows affected, 1 warning (0.02 sec)
MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)
MariaDB [(none)]> exit
Bye
Create a MySQL schema file, named powerdns-schema.sql
,
with the following content:
/* SQL statements to create tables in designate_pdns DB. Note: This file is taken as is from: https://raw.githubusercontent.com/openstack/designate/master/devstack/designate_plugins/backend-pdns4-mysql-db.sql */ CREATE TABLE domains ( id INT AUTO_INCREMENT, name VARCHAR(255) NOT NULL, master VARCHAR(128) DEFAULT NULL, last_check INT DEFAULT NULL, type VARCHAR(6) NOT NULL, notified_serial INT DEFAULT NULL, account VARCHAR(40) DEFAULT NULL, PRIMARY KEY (id) ) Engine=InnoDB; CREATE UNIQUE INDEX name_index ON domains(name); CREATE TABLE records ( id INT AUTO_INCREMENT, domain_id INT DEFAULT NULL, name VARCHAR(255) DEFAULT NULL, type VARCHAR(10) DEFAULT NULL, -- Changed to "TEXT", as VARCHAR(65000) is too big for most MySQL installs content TEXT DEFAULT NULL, ttl INT DEFAULT NULL, prio INT DEFAULT NULL, change_date INT DEFAULT NULL, disabled TINYINT(1) DEFAULT 0, ordername VARCHAR(255) BINARY DEFAULT NULL, auth TINYINT(1) DEFAULT 1, PRIMARY KEY (id) ) Engine=InnoDB; CREATE INDEX nametype_index ON records(name,type); CREATE INDEX domain_id ON records(domain_id); CREATE INDEX recordorder ON records (domain_id, ordername); CREATE TABLE supermasters ( ip VARCHAR(64) NOT NULL, nameserver VARCHAR(255) NOT NULL, account VARCHAR(40) NOT NULL, PRIMARY KEY (ip, nameserver) ) Engine=InnoDB; CREATE TABLE comments ( id INT AUTO_INCREMENT, domain_id INT NOT NULL, name VARCHAR(255) NOT NULL, type VARCHAR(10) NOT NULL, modified_at INT NOT NULL, account VARCHAR(40) NOT NULL, -- Changed to "TEXT", as VARCHAR(65000) is too big for most MySQL installs comment TEXT NOT NULL, PRIMARY KEY (id) ) Engine=InnoDB; CREATE INDEX comments_domain_id_idx ON comments (domain_id); CREATE INDEX comments_name_type_idx ON comments (name, type); CREATE INDEX comments_order_idx ON comments (domain_id, modified_at); CREATE TABLE domainmetadata ( id INT AUTO_INCREMENT, domain_id INT NOT NULL, kind VARCHAR(32), content TEXT, PRIMARY KEY (id) ) Engine=InnoDB; CREATE INDEX domainmetadata_idx ON domainmetadata (domain_id, kind); CREATE TABLE cryptokeys ( id INT AUTO_INCREMENT, domain_id INT NOT NULL, flags INT NOT NULL, active BOOL, content TEXT, PRIMARY KEY(id) ) Engine=InnoDB; CREATE INDEX domainidindex ON cryptokeys(domain_id); CREATE TABLE tsigkeys ( id INT AUTO_INCREMENT, name VARCHAR(255), algorithm VARCHAR(50), secret VARCHAR(255), PRIMARY KEY (id) ) Engine=InnoDB; CREATE UNIQUE INDEX namealgoindex ON tsigkeys(name, algorithm);
Create the PowerDNS schema for the database using mysql
CLI. For example:
root #
mysql powerdns < powerdns-schema.sql
Enable pdns
systemd service.
root #
systemctl enable pdnsroot #
systemctl start pdns
If pdns
is successfully running, you should see the
following logs by running journalctl -u pdns
command.
Feb 07 01:44:12 d52-54-77-77-01-01 systemd[1]: Started PowerDNS Authoritative Server. Feb 07 01:44:12 d52-54-77-77-01-01 pdns_server[21285]: Done launching threads, ready to distribute questions
Configure Designate to use PowerDNS backend by appending the PowerDNS
servers to /etc/designate/pools.crowbar.yaml
file
on a Designate worker node.
If we are replacing Bind9 backend with PowerDNS backend, make sure to
remove the bind9
entries from
/etc/designate/pools.crowbar.yaml
.
In HA deployment, there should be multiple PowerDNS entries.
Also, make sure the api_token
matches the
api-key
that was specified in the
/etc/pdns/pdns.conf
file earlier.
Append the PowerDNS entries to the end of
/etc/designate/pools.crowbar.yaml
. For example:
--- - name: default-bind description: Default BIND9 Pool id: 794ccc2c-d751-44fe-b57f-8894c9f5c842 attributes: {} ns_records: - hostname: public-d52-54-77-77-01-01.virtual.cloud.suse.de. priority: 1 - hostname: public-d52-54-77-77-01-02.virtual.cloud.suse.de. priority: 1 nameservers: - host: 192.168.124.83 port: 53 - host: 192.168.124.81 port: 53 also_notifies: [] targets: - type: bind9 description: BIND9 Server masters: - host: 192.168.124.83 port: 5354 - host: 192.168.124.82 port: 5354 - host: 192.168.124.81 port: 5354 options: host: 192.168.124.83 port: 53 rndc_host: 192.168.124.83 rndc_port: 953 rndc_key_file: "/etc/designate/rndc.key" - type: bind9 description: BIND9 Server masters: - host: 192.168.124.83 port: 5354 - host: 192.168.124.82 port: 5354 - host: 192.168.124.81 port: 5354 options: host: 192.168.124.81 port: 53 rndc_host: 192.168.124.81 rndc_port: 953 rndc_key_file: "/etc/designate/rndc.key" - type: pdns4 description: PowerDNS4 DNS Server masters: - host: 192.168.124.83 port: 5354 - host: 192.168.124.82 port: 5354 - host: 192.168.124.81 port: 5354 options: host: 192.168.124.83 port: 54 api_endpoint: http://192.168.124.83:8081 api_token: Sfw234sDFw90z
Update the pools using designate-manage
CLI.
tux >
designate-manage pool update --file /etc/designate/pools.crowbar.yaml
Once Designate sync up with PowerDNS, you should see the domains in the PowerDNS database which reflects the zones in Designate.
It make take a few minutes for Designate to sync with PowerDNS.
We can verify that the domains are successfully sync up with Designate
by inpsecting the domains
table in the database.
For example:
root #
mysql powerdns
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 21131
Server version: 10.2.29-MariaDB-log SUSE package
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [powerdns]> select * from domains;
+----+---------+--------------------------------------------------------------+------------+-------+-----------------+---------+
| id | name | master | last_check | type | notified_serial | account |
+----+---------+--------------------------------------------------------------+------------+-------+-----------------+---------+
| 1 | foo.bar | 192.168.124.81:5354 192.168.124.82:5354 192.168.124.83:5354 | NULL | SLAVE | NULL | |
+----+---------+--------------------------------------------------------------+------------+-------+-----------------+---------+
1 row in set (0.00 sec)
To make the SUSE OpenStack Cloud controller functions and the Compute Nodes highly available, set up one or more clusters by deploying Pacemaker (see Section 2.6, “High Availability” for details). Since it is possible (and recommended) to deploy more than one cluster, a separate proposal needs to be created for each cluster.
Deploying Pacemaker is optional. In case you do not want to deploy it, skip this section and start the node deployment by deploying the database as described in Section 12.3, “Deploying the Database”.
To set up a cluster, at least two nodes are required. See Section 2.6.5, “Cluster Requirements and Recommendations” for more information.
To create a proposal, go to
› and click for the Pacemaker barclamp. A drop-down box where you can enter a name and a description for the proposal opens. Click to open the configuration screen for the proposal.The name you enter for the proposal will be used to generate host names for the virtual IP addresses of HAProxy. By default, the names follow this scheme:
cluster-PROPOSAL_NAME.FQDN
(for the internal name) |
public-cluster-PROPOSAL_NAME.FQDN
(for the public name) |
For example, when PROPOSAL_NAME is set to
data
, this results in the following names:
cluster-data.example.com
|
public-cluster-data.example.com
|
For requirements regarding SSL encryption and certificates, see Section 2.3, “SSL Encryption”.
The following options are configurable in the Pacemaker configuration screen:
Choose a technology used for cluster communication. You can choose between
, sending a message to multiple destinations, or , sending a message to a single destination. By default unicast is used.Whenever communication fails between one or more nodes and the rest of the cluster a “cluster partition” occurs. The nodes of a cluster are split in partitions but are still active. They can only communicate with nodes in the same partition and are unaware of the separated nodes. The cluster partition that has the majority of nodes is defined to have “quorum”.
This configuration option defines what to do with the cluster partition(s) that do not have the quorum. See https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-config-basics-global-quorum, for details.
The recommended setting is to choose
. However, is enforced for two-node clusters to ensure that the remaining node continues to operate normally in case the other node fails. For clusters using shared resources, choosing may be used to ensure that these resources continue to be available.“Misbehaving” nodes in a cluster are shut down to prevent them from causing trouble. This mechanism is called STONITH (“Shoot the other node in the head”). STONITH can be configured in a variety of ways, refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#cha-ha-fencing for details. The following configuration options exist:
STONITH will not be configured when deploying the barclamp. It needs to be configured manually as described in https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#cha-ha-fencing. For experts only.
Using this option automatically sets up STONITH with data received from the IPMI barclamp. Being able to use this option requires that IPMI is configured for all cluster nodes. This should be done by default. To check or change the IPMI deployment, go to
› › › . Also make sure the option is set to on this barclamp.To configure STONITH with the IPMI data, all STONITH devices must support IPMI. Problems with this setup may occur with IPMI implementations that are not strictly standards compliant. In this case it is recommended to set up STONITH with STONITH block devices (SBD).
This option requires manually setting up shared storage and a watchdog on the cluster nodes before applying the proposal. To do so, proceed as follows:
Prepare the shared storage. The path to the shared storage device must be persistent and consistent across all nodes in the cluster. The SBD device must not use host-based RAID or cLVM2.
Install the package sbd
on
all cluster nodes.
Initialize the SBD device with by running the following command.
Make sure to replace
/dev/SBD
with the
path to the shared storage device.
sbd -d /dev/SBD create
Refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#pro-ha-storage-protect-sbd-create for details.
In
, specify the respective kernel module to be used. Find the most commonly used watchdog drivers in the following table:Hardware | Driver |
---|---|
HP | hpwdt
|
Dell, Fujitsu, Lenovo (Intel TCO) | iTCO_wdt
|
Xen VM (DomU) | xen_xdt
|
Generic | softdog
|
If your hardware is not listed above, either ask your hardware vendor
for the right name or check the following directory for a list of
choices:
/lib/modules/KERNEL_VERSION/kernel/drivers/watchdog
.
Alternatively, list the drivers that have been installed with your kernel version:
root #
rpm
-ql kernel-VERSION |grep
watchdog
If the nodes need different watchdog modules, leave the text box empty.
After the shared storage has been set up, specify the path using the
“by-id” notation
(/dev/disk/by-id/DEVICE
).
It is possible to specify multiple paths as a comma-separated list.
Deploying the barclamp will automatically complete the SBD setup on the cluster nodes by starting the SBD daemon and configuring the fencing resource.
All nodes will use the identical configuration. Specify the
to use and enter for the agent.
To get a list of STONITH devices which are supported by the High
Availability Extension, run the following command on an already
installed cluster nodes: stonith -L
. The list of
parameters depends on the respective agent. To view a list of
parameters use the following command:
stonith -t agent -n
All nodes in the cluster use the same
, but can be configured with different parameters. This setup is, for example, required when nodes are in different chassis and therefore need different IPMI parameters.
To get a list of STONITH devices which are supported by the High
Availability Extension, run the following command on an already
installed cluster nodes: stonith -L
. The list of
parameters depends on the respective agent. To view a list of
parameters use the following command:
stonith -t agent -n
Use this setting for completely virtualized test installations. This option is not supported.
With STONITH, Pacemaker clusters with two nodes may sometimes hit an issue known as STONITH deathmatch where each node kills the other one, resulting in both nodes rebooting all the time. Another similar issue in Pacemaker clusters is the fencing loop, where a reboot caused by STONITH will not be enough to fix a node and it will be fenced again and again.
This setting can be used to limit these issues. When set to OpenStack Cloud operator. When set to , the services for Pacemaker will always be started on boot. The value is used to have the most appropriate value automatically picked: it will be for two-node clusters (to avoid STONITH deathmatches), and otherwise.
, a node that has not been properly shut down or rebooted will not start the services for Pacemaker on boot. Instead, the node will wait for action from the SUSE
When a node boots but not starts corosync because of this setting, then
the node's status is in the Problem
" (red dot).
Get notified of cluster node failures via e-mail. If set to
, you need to specify which to use, a prefix for the mails' subject and sender and recipient addresses. Note that the SMTP server must be accessible by the cluster nodes.The public name is the host name that will be used instead of the generated public name (see Important: Proposal Name) for the public virtual IP address of HAProxy. (This is the case when registering public endpoints, for example). Any name specified here needs to be resolved by a name server placed outside of the SUSE OpenStack Cloud network.
The Pacemaker component consists of the following roles. Deploying the
role is optional:Deploy this role on all nodes that should become member of the cluster.
Deploying this role is optional. If deployed, sets up the Hawk Web
interface which lets you monitor the status of the cluster. The Web
interface can be accessed via
https://IP-ADDRESS:7630
. The
default hawk credentials are username hacluster
, password
crowbar
.
The password is visible and editable in the "corosync":
section of the
view.
Note that the GUI on SUSE OpenStack Cloud can only be used to monitor the cluster status and not to change its configuration.
may be deployed on at least one cluster node. It is recommended to deploy it on all cluster nodes.
Deploy this role on all nodes that should become members of the
Compute Nodes cluster. They will run as Pacemaker remote nodes that are
controlled by the cluster, but do not affect quorum. Instead of the
complete cluster stack, only the pacemaker-remote
component will be installed on this nodes.
After a cluster has been successfully deployed, it is listed under
in the section and can be used for role deployment like a regular node.When using clusters, roles from other barclamps must never be deployed to single nodes that are already part of a cluster. The only exceptions from this rule are the following roles:
cinder-volume
swift-proxy + swift-dispersion
swift-ring-compute
swift-storage
After a role has been deployed on a cluster, its services are managed by the HA software. You must never manually start or stop an HA-managed service, nor configure it to start on boot. Services may only be started or stopped by using the cluster management tools Hawk or the crm shell. See https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-config-basics-resources for more information.
To check whether all cluster resources are running, either use the Hawk
Web interface or run the command crm_mon
-1r
. If it is not the case, clean up the respective
resource with crm
resource
cleanup
RESOURCE , so it gets
respawned.
Also make sure that STONITH correctly works before continuing with the SUSE OpenStack Cloud setup. This is especially important when having chosen a STONITH configuration requiring manual setup. To test if STONITH works, log in to a node on the cluster and run the following command:
pkill -9 corosync
In case STONITH is correctly configured, the node will reboot.
Before testing on a production cluster, plan a maintenance window in case issues should arise.
The very first service that needs to be deployed is the
. The database component is using MariaDB and is used by all other components. It must be installed on a Control Node. The Database can be made highly available by deploying it on a cluster.The only attribute you may change is the maximum number of database connections (
). The default value should usually work—only change it for large deployments in case the log files show database connection failures.Deploying the database requires the use of MariaDB
MariaDB back end features full HA support based on the Galera clustering technology. The HA setup requires an odd number of nodes. The recommended number of nodes is 3.
SSL can be enabled with either a stand-alone or cluster deployment. The replication traffic between database nodes is not encrypted, whilst traffic between the database server(s) and clients are, so a separate network for the database servers is recommended.
Certificates can be provided, or the barcamp can generate self-signed
certificates. The certificate filenames are configurable in the
barclamp, and the directories /etc/mysql/ssl/certs
and /etc/mysql/ssl/private
to use the defaults will
need to be created before the barclamp is applied. The CA certificate
and the certificate for MariaDB to use both go into
/etc/mysql/ssl/certs
. The appropriate private key
for the certificate is placed into the
/etc/mysql/ssl/private
directory. As long as the
files are readable when the barclamp is deployed, permissions can be
tightened after a successful deployment once the appropriate UNIX
groups exist.
The Common Name (CN) for the SSL certificate must be fully
qualified server name
for single host deployments, and
cluster-cluster name
.full domain
name
for cluster deployments.
If certificate validation errors are causing issues with deploying
other barclamps (for example, when creating databases or users) you
can check the configuration with
mysql --ssl-verify-server-cert
which will perform
the same verification that Crowbar does when connecting to the
database server.
If certificates are supplied, the CA certificate and its full trust
chain must be in the ca.pem
file. The certificate
must be trusted by the machine (or all cluster members in a cluster
deployment), and it must be available on all client machines —
IE, if the OpenStack services are deployed on separate machines or
cluster members they will all require the CA certificate to be in
/etc/mysql/ssl/certs
as well as trusted by the
machine.
The following configuration settings are available via the
barclamp graphical interface:Path to a directory for storing database data.
The maximum number of simultaneous client connections.
A period after which the binary logs are removed.
When enabled, all queries that take longer than usual to execute
are logged to a separate log file (by default, it's
/var/log/mysql/mysql_slow.log
). This can be
useful for debugging.
When MariaDB is used as the database back end, the
role cannot be deployed to the node with the role. These two roles cannot coexist due to the fact that Monasca uses its own MariaDB instance.The RabbitMQ messaging system enables services to communicate with the other nodes via Advanced Message Queue Protocol (AMQP). Deploying it is mandatory. RabbitMQ needs to be installed on a Control Node. RabbitMQ can be made highly available by deploying it on a cluster. We recommend not changing the default values of the proposal's attributes.
Name of the default virtual host to be created and used by the RabbitMQ
server (default_vhost
configuration option in
rabbitmq.config
).
Port the RabbitMQ server listens on (tcp_listeners
configuration option in rabbitmq.config
).
RabbitMQ default user (default_user
configuration
option in rabbitmq.config
).
To make RabbitMQ highly available, deploy it on a cluster instead of a single Control Node. This also requires shared storage for the cluster that hosts the RabbitMQ data. We recommend using a dedicated cluster to deploy RabbitMQ together with the database, since both components require shared storage.
Deploying RabbitMQ on a cluster makes an additional
section available in the section of the proposal. Configure the in this section.The RabbitMQ barclamp supports securing traffic via SSL. This is similar to the SSL support in other barclamps, but with these differences:
RabbitMQ can listen on two ports at the same time, typically port 5672 for unsecured and port 5671 for secured traffic.
The Ceilometer pipeline for OpenStack Swift cannot be passed SSL-related parameters. When SSL is enabled for RabbitMQ the Ceilometer pipeline in Swift is turned off, rather than sending it over an unsecured channel.
The following steps are the fastest way to set up and test a new SSL certificate authority (CA).
In the RabbitMQ barclamp set true
, then apply the barclamp. The barclamp will create a new CA, enter the correct settings in /etc/rabbitmq/rabbitmq.config
, and start RabbitMQ.
Test your new CA with OpenSSL, substituting the hostname of your control node:
openssl s_client -connect d52-54-00-59-e5-fd:5671 [...] Verify return code: 18 (self signed certificate)
This outputs a lot of information, including a copy of the server's public certificate, protocols, ciphers, and the chain of trust.
The last step is to configure client services to use SSL to access the RabbitMQ service. (See https://docs.openstack.org/oslo.messaging/pike/#oslo-messaging-rabbit for a complete reference).
It is preferable to set up your own CA. The best practice is to use a commercial certificate authority. You may also deploy your own self-signed certificates, provided that your cloud is not publicly-accessible, and only for your internal use. Follow these steps to enable your own CA in RabbitMQ and deploy it to SUSE OpenStack Cloud:
Configure the RabbitMQ barclamp to use the control node's certificate authority (CA), if it already has one, or create a CA specifically for RabbitMQ and configure the barclamp to use that. (See Section 2.3, “SSL Encryption”, and the RabbitMQ manual has a detailed howto on creating your CA at http://www.rabbitmq.com/ssl.html, with customizations for .NET and Java clients.)
The configuration options in the RabbitMQ barclamp allow tailoring the barclamp to your SSL setup.
Set this to
to expose all of your configuration options.RabbitMQ's SSL listening port. The default is 5671.
When this is set to true
, self-signed certificates are automatically generated and copied to the correct locations on the control node, and all other barclamp options are set automatically. This is the fastest way to apply and test the barclamp. Do not use this on production systems. When this is set to false
the remaining options are exposed.
The location of your public root CA certificate.
The location of your private server key.
This goes with
. Set to to require clients to present SSL certificates to RabbitMQ.Trust client certificates presented by the clients that are signed by other CAs. You'll need to store copies of the CA certificates; see "Trust the Client's Root CA" at http://www.rabbitmq.com/ssl.html.
When this is set to
, clients validate the RabbitMQ server certificate with the file.Tells clients of RabbitMQ where to find the CA bundle that validates the certificate presented by the RabbitMQ server, when
is set to .
RabbitMQ has an option called Configure clients to send
notifications
. It defaults to false
, which
means no events will be sent. It is required to be set to
true
for Ceilometer, Monasca, and any other services
consuming notifications. When it is set to true
,
OpenStack services are configured to submit lifecycle audit events to the
notification
RabbitMQ queue.
This option should only be enabled if an active consumer is configured, otherwise events will accumulate on the RabbitMQ server, clogging up CPU, memory, and disk storage.
Any accumulation can be cleared by running:
$ rabbitmqctl -p /openstack purge_queue notifications.info $ rabbitmqctl -p /openstack purge_queue notifications.error
Keystone is another core component that is used by all other OpenStack components. It provides authentication and authorization services. Keystone needs to be installed on a Control Node. Keystone can be made highly available by deploying it on a cluster. You can configure the following parameters of this barclamp:
Set the algorithm used by Keystone to generate the tokens. You can
choose between Fernet
(the default) or
UUID
. Note that for performance and security reasons
it is strongly recommended to use Fernet
.
Allows customizing the region name that crowbar is going to manage.
Tenant for the users. Do not change the default value of
openstack
.
User name and password for the administrator.
Specify whether a regular user should be created automatically. Not recommended in most scenarios, especially in an LDAP environment.
User name and password for the regular user. Both the regular user and the administrator accounts can be used to log in to the SUSE OpenStack Cloud Dashboard. However, only the administrator can manage Keystone users and access.
When you use the default value Section 2.3, “SSL Encryption” for background information and Section 11.4.6, “Enabling SSL” for installation instructions. The following additional configuration options will become available when choosing :
, public communication will not be encrypted. Choose to use SSL for encryption. See
When set to true
, self-signed certificates are
automatically generated and copied to the correct locations. This
setting is for testing purposes only and should never be used in
production environments!
Location of the certificate key pair files.
Set this option to true
when using self-signed
certificates to disable certificate checks. This setting is for
testing purposes only and should never be used in production
environments!
Specify the absolute path to the CA certificate. This field is
mandatory, and leaving it blank will cause the barclamp to fail. To fix
this issue, you have to provide the absolute path to the CA
certificate, restart the apache2
service, and
re-deploy the barclamp.
When the certificate is not already trusted by the pre-installed list of trusted root certificate authorities, you need to provide a certificate bundle that includes the root and all intermediate CAs.
Keystone has the ability to separate identity backends by domains. SUSE OpenStack Cloud 8 uses this method for authenticating users.
The Keystone barclamp sets up a MariaDB database by default. Configuring an LDAP back-end is done in the
view.Set
Then in the
section configure a map with domain names as keys, and configuration as values. In the default proposal the domain name key is , and the keys are the two required sections for an LDAP-based identity driver configuration, the section which sets the driver, and the section which sets the LDAP connection options. You may configure multiple domains, each with its own configuration.You may make this available to Horizon by setting
to in the Horizon barclamp.Users in the LDAP-backed domain have to know the name of the domain in order to authenticate, and must use the Keystone v3 API endpoint. (See the OpenStack manuals, Domain-specific Configuration and Integrate Identity with LDAP, for additional details.)
Making Keystone highly available requires no special configuration—it is sufficient to deploy it on a cluster.
Monasca is an open-source monitoring-as-a-service solution that integrates with OpenStack. Monasca is designed for scalability, high performance, and fault tolerance.
Accessing the
interface is not required for day-to-day operation. But as not all Monasca settings are exposed in the barclamp graphical interface (for example, various performance tuneables), it is recommended to configure Monasca in the mode. Below are the options that can be configured via the interface of the Monasca barclamp.Contains Keystone credentials that the agents use to send metrics. Do not change these options, as they are configured by Crowbar.
Specifies whether SSL certificates are verified when communicating with
Keystone. If set to false
, the
ca_file
option must be specified.
Specifies the location of a CA certificate that is used for verifying Keystone's SSL certificate.
Path for storing log files. The specified path must exist. Do not change
the default /var/log/monasca-agent
path.
Agent's log level. Limits log messages to the specified level and above. The following levels are available: Error, Warning, Info (default), and Debug.
Interval in seconds between running agents' checks.
Number of simultaneous collector threads to run. This refers to the
maximum number of different collector plug-ins (for example,
http_check
) that are allowed to run simultaneously. The
default value 1
means that plug-ins are run
sequentially.
If a problem with the results from multiple plug-ins results blocks the
entire thread pool (as specified by the
num_collector_threads
parameter), the collector
exits, so it can be restarted by the
supervisord
. The parameter
pool_full_max_retries
specifies when this event
occurs. The collector exits when the defined number of consecutive
collection cycles have ended with the thread pool completely full.
Upper limit in seconds for any collection plug-in's run time. A warning is logged if a plug-in runs longer than the specified limit.
Maximum number of measurements to buffer locally if the Monasca API
is unreachable. Measurements will be dropped in batches, if the API is
still unreachable after the specified number of messages are buffered.
The default -1
value indicates unlimited buffering.
Note that a large buffer increases the agent's memory usage.
Maximum number of measurements to send when the local measurement buffer is flushed.
Number of extra dimensions to add to metrics sent to the Monasca API.
This option is intended for load testing purposes only. Do not enable the
option in production! The default 0
value disables the
addition of dimensions.
Maximum payload size in kilobytes for a request sent to the Monasca log API.
Maximum number of log entries the log agent sends to the Monasca log API in a single request. Reducing the number increases performance.
Time interval in seconds between sending logs to the Monasca log API.
Interval in seconds for checking whether
elapsed_time_sec
has been reached.
Keystone credentials the log agents use to send logs to the Monasca log API. Do not change this option manually, as it is configured by Crowbar.
Interfaces monasca-api
listens on. Do not change this
option, as it is configured by Crowbar.
Number of processes to spawn.
Number of WSGI worker threads to spawn.
Log level for openstack-monasca-api
. Limits log
messages to the specified level and above. The following levels are
available: Critical, Error, Warning, Info (default), Debug, and Trace.
List of directories for storing elasticsearch snapshots. Must be created
manually and be writeable by the
elasticsearch
user.
Must contain at least one entry in order for the snapshot functionality
to work.
For instructions on creating an elasticsearch snapshot, see https://documentation.suse.com/soc/8/html/suse-openstack-cloud-socmmsoperator/idg-msoperator-shared-operationmaintenance-c-operate-xml-1.html.
elasticsearch-curator
removes old and large
elasticsearch indices. The settings below determine its behavior.
Time threshold for deleting indices. Indices older the specified number of days are deleted. This parameter is unset by default, so indices are kept indefinitely.
Maximum size in megabytes of indices. Indices larger than the specified size are deleted. This parameter is unset by default, so indices are kept irrespective of their size.
List of indices to exclude from
elasticsearch-curator
runs. By default, only the
.kibana
files are excluded.
Specifies when to run elasticsearch-curator
.
Attributes of this parameter correspond to the fields in
crontab(5)
.
Number of hours for retaining log segments in Kafka's on-disk log. Messages older than the specified value are dropped.
Maximum size for Kafka's on-disk log in bytes. If the log grows beyond this size, the oldest log segments are dropped.
Number of days to keep metrics records in influxdb.
For an overview of all supported values, see https://docs.influxdata.com/influxdb/v1.1/query_language/database_management/#create-retention-policies-with-create-retention-policy.
Enable or disable email alarm notifications.
SMTP smarthost for sending alarm notifications.
Port for the SMTP smarthost.
User name for authenticating against the smarthost.
Password for authenticating against the smarthost.
Sender address for alarm notifications.
The global switch for toggling libvirt monitoring. If set to true, libvirt metrics will be gathered on all libvirt based Compute Nodes. This setting is available in the Crowbar UI.
The global switch for toggling Ceph monitoring. If set to true, Ceph metrics will be gathered on all Ceph-based Compute Nodes. This setting is available in Crowbar UI. If the Ceph cluster has been set up independently, Crowbar ignores this setting.
The directory where monasca-agent will locally cache various metadata about locally running VMs on each Compute Node.
Specifies the list of instance metadata keys to be included as dimensions with customer metrics. This is useful for providing more information about an instance.
Specifies a minimum interval in seconds for collecting disk metrics.
Increase this value to reduce I/O load. If the value is lower than the
global agent collection period (check_frequency
), it
will be ignored in favor of the global collection period.
Specifies the number of ping command processes to run concurrently when
determining whether the VM is reachable. This should be set to a value
that allows the plug-in to finish within the agent's collection period,
even if there is a networking issue. For example, if the expected number
of VMs per Compute Node is 40 and each VM has one IP address, then the
plug-in will take at least 40 seconds to do the ping checks in the
worst-case scenario where all pings fail (assuming the default timeout of
1 second). Increasing max_ping_concurrency
allows the
plug-in to finish faster.
Specifies the list of Nova side instance metadata keys to be included as dimensions with the cross-tenant metrics for the
project. This is useful for providing more information about an instance.Specifies the number of seconds between calls to the Nova API to refresh the instance cache. This is helpful for updating VM hostname and pruning deleted instances from the cache. By default, it is set to 14,400 seconds (four hours). Set to 0 to refresh every time the Collector runs, or to None to disable regular refreshes entirely. In this case, the instance cache will only be refreshed when a new instance is detected.
Includes the entire ping command (without the IP address, which is
automatically appended) to perform a ping check against instances. The
NAMESPACE
keyword is automatically replaced with the
appropriate network namespace for the VM being monitored. Set to
False to disable ping checks.
Specifies a minimum interval in seconds for collecting disk metrics.
Increase this value to reduce I/O load. If the value is lower than the
global agent collection period (check_frequency
), it
will be ignored in favor of the global collection period.
Toggles the collection of VM CPU metrics. Set to true to enable.
Toggles the collection of VM disk metrics. Set to true to enable.
Toggles the collection of extended disk metrics. Set to true to enable.
Toggles the collection of VM network metrics. Set to true to enable.
Toggles ping checks for checking whether a host is alive. Set to true to enable.
Specifies a period of time (in seconds) in which to suspend metrics from a newly-created VM. This is to prevent quickly-obsolete metrics in an environment with a high amount of instance churn (VMs created and destroyed in rapid succession). The default probation length is 300 seconds (5 minutes). Set to 0 to disable VM probation. In this case, metrics are recorded immediately after a VM is created.
Specifies a minimum interval in seconds for collecting VM network metrics. Increase this value to reduce I/O load. If the value is lower than the global agent collection period (check_frequency), it will be ignored in favor of the global collection period.
The Monasca component consists of following roles:
Monasca server-side components that are deployed by Chef.
Currently, this only creates Keystone resources required by Monasca,
such as users, roles, endpoints, etc. The rest is left to the
Ansible-based monasca-installer
run by the
monasca-master
role.
Runs the Ansible-based monasca-installer
from
the Crowbar node. The installer deploys the Monasca server-side
components to the node that has the
monasca-server
role assigned to it. These
components are openstack-monasca-api
, and
openstack-monasca-log-api
, as well as all the
back-end services they use.
Deploys openstack-monasca-agent
that is
responsible for sending metrics to monasca-api
on nodes it is assigned to.
Deploys openstack-monasca-log-agent
responsible
for sending logs to monasca-log-api
on nodes it
is assigned to.
Swift adds an object storage service to SUSE OpenStack Cloud for storing single files such as images or snapshots. It offers high data security by storing the data redundantly on a pool of Storage Nodes—therefore Swift needs to be installed on at least two dedicated nodes.
To properly configure Swift it is important to understand how it places the data. Data is always stored redundantly within the hierarchy. The Swift hierarchy in SUSE OpenStack Cloud is formed out of zones, nodes, hard disks, and logical partitions. Zones are physically separated clusters, for example different server rooms each with its own power supply and network segment. A failure of one zone must not affect another zone. The next level in the hierarchy are the individual Swift storage nodes (on which has been deployed), followed by the hard disks. Logical partitions come last.
Swift automatically places three copies of each object on the highest hierarchy level possible. If three zones are available, then each copy of the object will be placed in a different zone. In a one zone setup with more than two nodes, the object copies will each be stored on a different node. In a one zone setup with two nodes, the copies will be distributed on different hard disks. If no other hierarchy element fits, logical partitions are used.
The following attributes can be set to configure Swift:
Set to true
to enable public access to containers.
If set to true, a copy of the current version is archived each time an object is updated.
Number of zones (see above). If you do not have different independent
installations of storage nodes, set the number of zones to
1
.
Partition power. The number entered here is used to compute the number of logical partitions to be created in the cluster. The number you enter is used as a power of 2 (2^X).
We recommend using a minimum of 100 partitions per disk. To measure the partition power for your setup, multiply the number of disks from all Swift nodes by 100, and then round up to the nearest power of two. Keep in mind that the first disk of each node is not used by Swift, but rather for the operating system.
Example: 10 Swift nodes with 5 hard disks each.
Four hard disks on each node are used for Swift, so there is a
total of forty disks. 40 x 100 = 4000. The nearest power of two, 4096,
equals 2^12. So the partition power that needs to be entered is
12
.
Changing the number of logical partition after Swift has been deployed is not supported. Therefore the value for the partition power should be calculated from the maximum number of partitions this cloud installation is likely going to need at any point in time.
This option sets the number of hours before a logical partition is
considered for relocation. 24
is the recommended
value.
The number of copies generated for each object. The number of replicas depends on the number of disks and zones.
Time (in seconds) after which to start a new replication process.
Shows debugging output in the log files when set to
true
.
Choose whether to encrypt public communication (
) or not ( ). If you choose , you have two options. You can either or provide the locations for the certificate key pair files. Using self-signed certificates is for testing purposes only and should never be used in production environments!Apart from the general configuration described above, the Swift barclamp lets you also activate and configure
. The features these middlewares provide can be used via the Swift command line client only. The Ratelimit and S3 middleware provide for the most interesting features, and we recommend enabling other middleware only for specific use-cases.Provides an S3 compatible API on top of Swift.
Serve container data as a static Web site with an index file and optional file listings. See http://docs.openstack.org/developer/swift/middleware.html#staticweb for details.
This middleware requires setting true
.
Create URLs to provide time-limited access to objects. See http://docs.openstack.org/developer/swift/middleware.html#tempurl for details.
Upload files to a container via Web form. See http://docs.openstack.org/developer/swift/middleware.html#formpost for details.
Extract TAR archives into a Swift account, and delete multiple objects or containers with a single request. See http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.bulk for details.
Interact with the Swift API via Flash, Java, and Silverlight from an external network. See http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.crossdomain for details.
Translates container and account parts of a domain to path parameters
that the Swift proxy server understands. Can be used to create
short URLs that are easy to remember, for example by rewriting
home.tux.example.com/$ROOT/tux/home/myfile
to home.tux.example.com/myfile
.
See
http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.domain_remap
for details.
Throttle resources such as requests per minute to provide denial of service protection. See http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.ratelimit for details.
The Swift component consists of four different roles. Deploying
is optional:The virtual object storage service. Install this role on all dedicated Swift Storage Nodes (at least two), but not on any other node.
Never install the swift-storage service on a node that runs other OpenStack components.
The ring maintains the information about the location of objects, replicas, and devices. It can be compared to an index that is used by various OpenStack components to look up the physical location of objects. must only be installed on a single node, preferably a Control Node.
The Swift proxy server takes care of routing requests to Swift. Installing a single instance of
on a Control Node is recommended. The role can be made highly available by deploying it on a cluster.Deploying
is optional. The Swift dispersion tools can be used to test the health of the cluster. It creates a heap of dummy objects (using 1% of the total space available). The state of these objects can be queried using the swift-dispersion-report query. needs to be installed on a Control Node.Swift replicates by design, so there is no need for a special HA setup. Make sure to fulfill the requirements listed in Section 2.6.4.1, “Swift—Avoiding Points of Failure”.
Glance provides discovery, registration, and delivery services for virtual disk images. An image is needed to start an instance—it is its pre-installed root-partition. All images you want to use in your cloud to boot instances from, are provided by Glance. Glance must be deployed onto a Control Node. Glance can be made highly available by deploying it on a cluster.
There are a lot of options to configure Glance. The most important ones are explained below—for a complete reference refer to http://github.com/crowbar/crowbar/wiki/Glance-barclamp.
As of SUSE OpenStack Cloud Crowbar 7, the Glance API v1 is no longer enabled by default. Instead, Glance API v2 is used by default.
If you need to re-enable API v1 for compatibility reasons:
Switch to the
view of the Glance barclamp.
Search for the enable_v1
entry and set it to
true
:
"enable_v1": true
In new installations, this entry is set to false
by
default. When upgrading from an older version of SUSE OpenStack Cloud Crowbar it is set
to true
by default.
Apply your changes.
Images are stored in an image file on the Control Node. .
Provides volume block storage to . SUSE OpenStack Cloud Crowbar. Use it to store images.
Provides an object storage service to . SUSE OpenStack Cloud Crowbar.
SUSE Enterprise Storage (based on Ceph) provides block storage service to . SUSE OpenStack Cloud Crowbar.
If you are using VMware as a hypervisor, it is recommended to use . for storing images. This will make starting VMware instances much faster.
If this is set to . , the API will communicate the direct URl of the image's back-end location to HTTP clients. Set to by default.
Depending on the storage back-end, there are additional configuration options available:
Only required if
is set to .Specify the directory to host the image file. The directory specified here can also be an NFS share. See Section 11.4.3, “Mounting NFS Shares on a Node” for more information.
Only required if
is set to .Set the name of the container to use for the images in Swift.
Only required if
is set to .If you are using an external Ceph cluster, specify the user you have set up for Glance (see Section 11.4.4, “Using an Externally Managed Ceph Cluster” for more information).
If you are using a SUSE OpenStack Cloud internal Ceph setup, the pool you specify here is created if it does not exist. If you are using an external Ceph cluster, specify the pool you have set up for Glance (see Section 11.4.4, “Using an Externally Managed Ceph Cluster” for more information).
Only required if
is set to .Name or IP address of the vCenter server.
vCenter login credentials.
A comma-separated list of datastores specified in the format: DATACENTER_NAME:DATASTORE_NAME
Specify an absolute path here.
Choose whether to encrypt public communication (SSL Support: Protocol for configuration details.
) or not ( ). If you choose , refer toEnable and configure image caching in this section. By default, image caching is disabled. You can see this the Raw view of your Nova barclamp:
image_cache_manager_interval = -1
This option sets the number of seconds to wait between runs of the image cache manager. Disabling it means that the cache manager will not automatically remove the unused images from the cache, so if you have many Glance images and are running out of storage you must manually remove the unused images from the cache. We recommend leaving this option disabled as it is known to cause issues, especially with shared storage. The cache manager may remove images still in use, e.g. when network outages cause synchronization problems with compute nodes.
If you wish to enable caching, re-enable it in a custom Nova
configuration file, for example
/etc/nova/nova.conf.d/500-nova.conf
. This sets the
interval to four minutes:
image_cache_manager_interval = 2400
See Chapter 14, Configuration Files for OpenStack Services for more information on custom configurations.
Learn more about Glance's caching feature at http://docs.openstack.org/developer/glance/cache.html.
Shows debugging output in the log files when set to
.Glance can be made highly available by deploying it on a cluster. We strongly recommended doing this for the image data as well. The recommended way is to use Swift or an external Ceph cluster for the image repository. If you are using a directory on the node instead (file storage back-end), you should set up shared storage on the cluster for it.
Cinder, the successor of Nova Volume, provides volume block storage. It adds persistent storage to an instance that will persist until deleted, contrary to ephemeral volumes that only persist while the instance is running.
Cinder can provide volume storage by using different back-ends such as local file, one or more local disks, Ceph (RADOS), VMware, or network storage solutions from EMC, EqualLogic, Fujitsu, NetApp or Pure Storage. Since SUSE OpenStack Cloud Crowbar 5, Cinder supports using several back-ends simultaneously. It is also possible to deploy the same network storage back-end multiple times and therefore use different installations at the same time.
The attributes that can be set to configure Cinder depend on the back-end. The only general option is SSL Support: Protocol for configuration details).
(seeWhen first opening the Cinder barclamp, the default proposal—
—is already available for configuration. To optionally add a back-end, go to the section and choose a from the drop-down box. Optionally, specify the . This is recommended when deploying the same volume type more than once. Existing back-end configurations (including the default one) can be deleted by clicking the trashcan icon if no longer needed. Note that you must configure at least one back-end.
Choose whether to use the “Available disks”
are all disks currently not used by the system. Note that one disk
(usually /dev/sda
) of every block storage node is
already used for the operating system and is not available for
Cinder.
Specify a name for the Cinder volume.
IP address and Port of the ECOM server.
Login credentials for the ECOM server.
VMAX port groups that expose volumes managed by this back-end.
Unique VMAX array serial number.
Unique pool name within a given array.
Name of the FAST Policy to be used. When specified, volumes managed by this back-end are managed as under FAST control.
For more information on the EMC driver refer to the OpenStack documentation at http://docs.openstack.org/liberty/config-reference/content/emc-vmax-driver.html.
EqualLogic drivers are included as a technology preview and are not supported.
Select the protocol used to connect, either
or .IP address and port of the ETERNUS SMI-S Server.
Login credentials for the ETERNUS SMI-S Server.
Storage pool (RAID group) in which the volumes are created. Make sure that the RAID group on the server has already been created. If a RAID group that does not exist is specified, the RAID group is built from unused disk drives. The RAID level is automatically determined by the ETERNUS DX Disk storage system.
For information on configuring the Hitachi HUSVM back-end, refer to http://docs.openstack.org/ocata/config-reference/block-storage/drivers/hitachi-storage-volume-driver.html.
SUSE OpenStack Cloud can use “Data ONTAP” in , or in . In vFiler will be configured, in vServer will be configured. The can be set to either or . Choose the driver and the protocol your NetApp is licensed for.
The management IP address for the 7-Mode storage controller, or the cluster management IP address for the clustered Data ONTAP.
Transport protocol for communicating with the storage controller or clustered Data ONTAP. Supported protocols are HTTP and HTTPS. Choose the protocol your NetApp is licensed for.
The port to use for communication. Port 80 is usually used for HTTP, 443 for HTTPS.
Login credentials.
The vFiler unit to be used for provisioning of OpenStack volumes. This setting is only available in .
Provide a list of comma-separated volume names to be used for provisioning. This setting is only available when using iSCSI as storage protocol.
A list of available file systems on an NFS server. Enter your NFS mountpoints in the
form in this format: host:mountpoint -o options. For example:host1:/srv/nfs/share1 /mnt/nfs/share1 -o rsize=8192,wsize=8192,timeo=14,intr
IP address of the FlashArray management VIP
API token for access to the FlashArray
Enable or disable iSCSI CHAP authentication
For more information on the Pure Storage FlashArray driver refer to the OpenStack documentation at https://docs.openstack.org/ocata/config-reference/block-storage/drivers/pure-storage-driver.html.
Select Section 11.4.4, “Using an Externally Managed Ceph Cluster” for setup instructions).
, if you are using an external Ceph cluster (seeName of the pool used to store the Cinder volumes.
Ceph user name.
Host name or IP address of the vCenter server.
vCenter login credentials.
Provide a comma-separated list of cluster names.
Path to the directory used to store the Cinder volumes.
Absolute path to the vCenter CA certificate.
Default value: false
(the CA truststore is used for
verification). Set this option to true
when using
self-signed certificates to disable certificate checks. This setting is
for testing purposes only and must not be used in production
environments!
Absolute path to the file to be used for block storage.
Maximum size of the volume file. Make sure not to overcommit the size, since it will result in data loss.
Specify a name for the Cinder volume.
Using a file for block storage is not recommended for production systems, because of performance and data security reasons.
Lets you manually pick and configure a driver. Only use this option for testing purposes, as it is not supported.
The Cinder component consists of two different roles:
The Cinder controller provides the scheduler and the API. Installing
on a Control Node is recommended.The virtual block storage service. It can be installed on a Control Node. However, we recommend deploying it on one or more dedicated nodes supplied with sufficient networking capacity to handle the increase in network traffic.
Both the
and the role can be deployed on a cluster.If you need to re-deploy
role from a single machine to a cluster environment, the following will happen: Volumes that are currently attached to instances will continue to work, but adding volumes to instances will not succeed.
To solve this issue, run the following script once on each node that
belongs to the /usr/bin/cinder-migrate-volume-names-to-cluster
.
The script is automatically installed by Crowbar on every machine or cluster that has a
role applied to it.In combination with Ceph or a network storage solution, deploying Cinder in a cluster minimizes the potential downtime. For
to be applicable to a cluster, the role needs all Cinder backends to be configured for non-local storage. If you are using local volumes or raw devices in any of your volume backends, you cannot apply to a cluster.Neutron provides network connectivity between interface devices managed by other OpenStack components (most likely Nova). The service works by enabling users to create their own networks and then attach interfaces to them.
Neutron must be deployed on a Control Node. You first need to choose a core plug-in—
or . Depending on your choice, more configuration options will become available.The
option lets you use an existing VMware installation. Using this plug-in is not a prerequisite for the VMware vSphere hypervisor support. For all other scenarios, choose .The only global option that can be configured is SSL Support: Protocol for configuration details.
. Choose whether to encrypt public communication ( ) or not ( ). If choosing , refer toSelect which mechanism driver(s) shall be enabled for the ml2 plug-in. It is possible to select more than one driver by holding the Ctrl key while clicking. Choices are:
Supports GRE, VLAN and VXLAN networks (to be configured via the . setting).
Supports VLANs only. Requires to specify the . .
Enables Neutron to dynamically adjust the VLAN settings of the ports of an existing Cisco Nexus switch when instances are launched. It also requires . which will automatically be selected. With , must be added. This option also requires to specify the . See Appendix B, Using Cisco Nexus Switches with Neutron for details.
vmware_dvs driver makes it possible to use Neutron for networking in a VMware-based environment. Choosing . , automatically selects the required , , and drivers. In the view, it is also possible to configure two additional attributes: (clean up the DVS portgroups on the target vCenter Servers when neutron-server is restarted) and (create DVS portgroups corresponding to networks in advance, rather than when virtual machines are attached to these networks).
With the default setup, all intra-Compute Node traffic flows through the network Control Node. The same is true for all traffic from floating IPs. In large deployments the network Control Node can therefore quickly become a bottleneck. When this option is set to “talk” to each other. Distributed Virtual Routers (DVR) require the driver and will not work with the driver. For details on DVR refer to https://wiki.openstack.org/wiki/Neutron/DVR.
, network agents will be installed on all compute nodes. This will de-centralize the network traffic, since Compute Nodes will be able to directlyThis option is only available when having chosen the Ctrl key while clicking.
or the mechanism drivers. Options are , and . It is possible to select more than one driver by holding the
When multiple type drivers are enabled, you need to select the
nova_fixed
network, that will be created when applying
the Neutron proposal. When manually creating provider networks with the
neutron
command, the default can be overwritten with
the --provider:network_type
type
switch. You will also need to
set a . It is
not possible to change this default when manually creating tenant
networks with the neutron
command. The non-default
type driver will only be used as a fallback.
Depending on your choice of the type driver, more configuration options become available.
Having chosen . , you also need to specify the start and end of the tunnel ID range.
The option . requires you to specify the .
Having chosen . , you also need to specify the start and end of the VNI range.
Neutron must not be deployed with the openvswitch with
gre
plug-in. See Appendix A, VMware vSphere Installation Instructions for details.
Host name or IP address of the xCAT Management Node.
xCAT login credentials.
List of rdev addresses that should be connected to this vswitch.
IP address of the xCAT management interface.
Net mask of the xCAT management interface.
The Neutron component consists of two different roles:
provides the scheduler and the API. It needs to be installed on a Control Node.
This service runs the various agents that manage the network traffic of all the cloud instances. It acts as the DHCP and DNS server and as a gateway for all cloud instances. It is recommend to deploy this role on a dedicated node supplied with sufficient network capacity.
In the Neutron barclamp, you can enable support for the infoblox IPAM
plug-in and configure it. For configuration, the
infoblox
section contains the subsections
grids
and grid_defaults
.
This subsection must contain at least one entry. For each entry, the following parameters are required:
admin_user_name
admin_password
grid_master_host
grid_master_name
data_center_name
You can also add multiple entries to the grids
section. However, the upstream infoblox agent only supports a single
grid currently.
This subsection contains the default settings that are used for each
grid (unless you have configured specific settings within the
grids
section).
For detailed information on all infoblox-related configuration settings, see https://github.com/openstack/networking-infoblox/blob/master/doc/source/installation.rst.
Currently, all configuration options for infoblox are only available in the
raw
mode of the Neutron barclamp. To enable support for
the infoblox IPAM plug-in and configure it, proceed as follows:
the Neutron barclamp proposal or create a new one.
Click
and search for the following section:"use_infoblox": false,
To enable support for the infoblox IPAM plug-in, change this entry to:
"use_infoblox": true,
In the grids
section, configure at least one grid by
replacing the example values for each parameter with real values.
If you need specific settings for a grid, add some of the parameters from
the grid_defaults
section to the respective grid entry
and adjust their values.
Otherwise Crowbar applies the default setting to each grid when you save the barclamp proposal.
Save your changes and apply them.
Neutron can be made highly available by deploying
and on a cluster. While may be deployed on a cluster shared with other services, it is strongly recommended to use a dedicated cluster solely for the role.This section shows you how to create external networks on SUSE OpenStack Cloud.
If you have not yet deployed Crowbar, add the following configuration to
/etc/crowbar/network.json
to set up an external network, using the name of your new network, VLAN
ID, and network addresses. If you have already deployed Crowbar, then add
this configuration to the view of the Network Barclamp.
"public2": { "conduit": "intf1", "vlan": 600, "use_vlan": true, "add_bridge": false, "subnet": "192.168.135.128", "netmask": "255.255.255.128", "broadcast": "192.168.135.255", "ranges": { "host": { "start": "192.168.135.129", "end": "192.168.135.254" } } },
Modify the additional_external_networks in the
view of the Neutron Barclamp with the name of your new external network.Apply both barclamps, and it may also be necessary to re-apply the Nova Barclamp.
Then follow the steps in the next section to create the new external network.
The following steps add the network settings, including IP address pools, gateway, routing, and virtual switches to your new network.
Set up interface mapping using either Open vSwitch (OVS) or Linuxbridge. For Open vSwitch run the following command:
neutron net-create public2 --provider:network_type flat \ --provider:physical_network public2 --router:external=True
For Linuxbridge run the following command:
neutron net-create --router:external True --provider:physical_network physnet1 \ --provider:network_type vlan --provider:segmentation_id 600
If a different network is used then Crowbar will create a new interface mapping. Then you can use a flat network:
neutron net-create public2 --provider:network_type flat \ --provider:physical_network public2 --router:external=True
Create a subnet:
neutron subnet-create --name public2 --allocation-pool \ start=192.168.135.2,end=192.168.135.127 --gateway 192.168.135.1 public2 \ 192.168.135.0/24 --enable_dhcp False
Create a router, router2:
neutron router-create router2
Connect router2 to the new external network:
neutron router-gateway-set router2 public2
Create a new private network and connect it to router2
neutron net-create priv-net neutron subnet-create priv-net --gateway 10.10.10.1 10.10.10.0/24 \ --name priv-net-sub neutron router-interface-add router2 priv-net-sub
Boot a VM on priv-net-sub and set a security group that allows SSH.
Assign a floating IP address to the VM, this time from network public2.
From the node verify that SSH is working by opening an SSH session to the VM.
For OVS, a new bridge will be created by Crowbar, in this case
br-public2
. In the bridge mapping the new network will
be assigned to the bridge. The interface specified in
/etc/crowbar/network.json
(in this case eth0.600) will
be plugged into br-public2
. The new public network can
be created in Neutron using the new public network name as
provider:physical_network.
For Linuxbridge, Crowbar will check the interface associated with public2. If this is the same as physnet1 no interface mapping will be created. The new public network can be created in Neutron using physnet1 as physical network and specifying the correct VLAN ID:
neutron net-create public2 --router:external True \ --provider:physical_network physnet1 --provider:network_type vlan \ --provider:segmentation_id 600
A bridge named brq-NET_ID
will be created and the
interface specified in /etc/crowbar/network.json
will
be plugged into it. If a new interface is associated in
/etc/crowbar/network.json
with
public2 then Crowbar will add a new interface
mapping and the second public network can be created using
public2 as the physical network:
neutron net-create public2 --provider:network_type flat \ --provider:physical_network public2 --router:external=True
Nova provides key services for managing the SUSE OpenStack Cloud, sets up the Compute Nodes. SUSE OpenStack Cloud currently supports KVM, Xen and VMware vSphere. The unsupported QEMU option is included to enable test setups with virtualized nodes. The following attributes can be configured for Nova:
Set the “overcommit ratio” for RAM for instances on the
Compute Nodes. A ratio of 1.0
means no overcommitment.
Changing this value is not recommended.
Set the “overcommit ratio” for CPUs for instances on the
Compute Nodes. A ratio of 1.0
means no overcommitment.
Set the “overcommit ratio” for virtual disks for instances
on the Compute Nodes. A ratio of 1.0
means no
overcommitment.
Amount of reserved host memory that is not used for allocating VMs by Nova Compute.
Allows to move KVM and Xen instances to a different Compute Node running the same hypervisor (cross hypervisor migrations are not supported). Useful when a Compute Node needs to be shut down or rebooted for maintenance or when the load of the Compute Node is very high. Instances can be moved while running (Live Migration).
Enabling the libvirt migration option will open a TCP port on the Compute Nodes that allows access to all instances from all machines in the admin network. Ensure that only authorized machines have access to the admin network when enabling this option.
It is possible to change a network to live migrate images. This is done
in the raw view of the Nova barclamp. In the
migration
section, change the
network
attribute to the appropriate value (for
example, storage
for Ceph).
Sets up a directory /var/lib/nova/instances
on the
Control Node on which is running. This
directory is exported via NFS to all compute nodes and will host a copy
of the root disk of all Xen instances. This setup
is required for live migration of Xen instances (but not for KVM)
and is used to provide central handling of instance data. Enabling this
option is only recommended if Xen live migration is
required—otherwise it should be disabled.
Setting up shared storage in a SUSE OpenStack Cloud where instances are running will result in connection losses to all running instances. It is strongly recommended to set up shared storage when deploying SUSE OpenStack Cloud. If it needs to be done at a later stage, make sure to shut down all instances prior to the change.
Kernel SamePage Merging (KSM) is a Linux Kernel feature which merges identical memory pages from multiple running processes into one memory region. Enabling it optimizes memory usage on the Compute Nodes when using the KVM hypervisor at the cost of slightly increasing CPU usage.
Setting up VMware support is described in a separate section. See Appendix A, VMware vSphere Installation Instructions.
Choose whether to encrypt public communication (SSL Support: Protocol for configuration details.
) or not ( ). If choosing ,refer to
Change the default VNC keymap for instances. By default,
en-us
is used. Enter the value in lowercase, either as
a two character code (such as de
or
jp
) or, as a five character code such as
de-ch
or en-uk
, if applicable.
After having started an instance you can display its VNC console in the OpenStack Dashboard (Horizon) via the browser using the noVNC implementation. By default this connection is not encrypted and can potentially be eavesdropped.
Enable encrypted communication for noVNC by choosing
and providing the locations for the certificate key pair files.Shows debugging output in the log files when set to
.You can pass custom vendor data to all VMs via Nova's metadata server. For example, information about a custom SMT server can be used by the SUSE guest images to automatically configure the repositories for the guest.
To pass custom vendor data, switch to the
view of the Nova barclamp.Search for the following section:
"metadata": { "vendordata": { "json": "{}" } }
As value of the json
entry, enter valid JSON data. For
example:
"metadata": { "vendordata": { "json": "{\"CUSTOM_KEY\": \"CUSTOM_VALUE\"}" } }
The string needs to be escaped because the barclamp file is in JSON format, too.
Use the following command to access the custom vendor data from inside a VM:
curl -s http://METADATA_SERVER/openstack/latest/vendor_data.json
The IP address of the metadata server is always the same from within a VM. For more details, see https://www.suse.com/communities/blog/vms-get-access-metadata-neutron/.
The Nova component consists of eight different roles:
Distributing and scheduling the instances is managed by the
. It also provides networking and messaging services. needs to be installed on a Control Node.
Provides the hypervisors (KVM, QEMU, VMware vSphere, Xen, and z/VM)
and tools needed to manage the instances. Only one hypervisor can be
deployed on a single compute node. To use different hypervisors in your
cloud, deploy different hypervisors to different Compute Nodes. A
nova-compute-*
role needs to be installed on every
Compute Node. However, not all hypervisors need to be deployed.
Each image that will be made available in SUSE OpenStack Cloud to start an instance
is bound to a hypervisor. Each hypervisor can be deployed on multiple
Compute Nodes (except for the VMware vSphere role, see below). In a
multi-hypervisor deployment you should make sure to deploy the
nova-compute-*
roles in a way, that enough compute
power is available for each hypervisor.
Existing nova-compute-*
nodes can be changed in a
production SUSE OpenStack Cloud without service interruption. You need to
“evacuate”
the node, re-assign a new nova-compute
role via the
Nova barclamp and the change.
can only be deployed on a single
node.
VMware vSphere is not supported “natively” by SUSE OpenStack Cloud—it rather delegates requests to an existing vCenter. It requires preparations at the vCenter and post install adjustments of the Compute Node. See Appendix A, VMware vSphere Installation Instructions for instructions. can only be deployed on a single Compute Node.
When deploying a
node with the ML2 driver enabled in the Neutron barclamp, the following new attributes are also available in the section of the mode: (the name of the DVS switch configured on the target vCenter cluster) and (enable or disable implementing security groups through DVS traffic rules).It is important to specify the correct he
value, as the barclamp expects the DVS switch to be preconfigured on the target VMware vCenter cluster.Deploying
nodes will not result in a functional cloud setup if the ML2 plug-in is not enabled in the Neutron barclamp.Making
highly available requires no special configuration—it is sufficient to deploy it on a cluster.To enable High Availability for Compute Nodes, deploy the following roles to one or more clusters with remote nodes:
nova-compute-kvm
nova-compute-qemu
nova-compute-xen
ec2-api
The cluster to which you deploy the roles above can be completely
independent of the one to which the role nova-controller
is deployed.
However, the nova-controller
and
ec2-api
roles must be deployed the same way (either
both to a cluster or both to
individual nodes. This is due to Crowbar design limitations.
It is recommended to use shared storage for the
/var/lib/nova/instances
directory, to ensure that
ephemeral disks will be preserved during recovery of VMs from failed
compute nodes. Without shared storage, any ephemeral disks will be lost,
and recovery will rebuild the VM from its original image.
If an external NFS server is used, enable the following option in the Nova barclamp proposal:
.The last component that needs to be deployed is Horizon, the OpenStack Dashboard. It provides a Web interface for users to start and stop instances and for administrators to manage users, groups, roles, etc. Horizon should be installed on a Control Node. To make Horizon highly available, deploy it on a cluster.
The following attributes can be configured:
Timeout (in minutes) after which a user is been logged out automatically. The default value is set to four hours (240 minutes).
Every Horizon session requires a valid Keystone token. These tokens also have a lifetime of four hours (14400 seconds). Setting the Horizon session timeout to a value larger than 240 will therefore have no effect, and you will receive a warning when applying the barclamp.
To successfully apply a timeout larger than four hours, you first need
to adjust the Keystone token expiration accordingly. To do so, open the
Keystone barclamp in token_expiration
. Note that the value has to
be provided in seconds. When the change is
successfully applied, you can adjust the Horizon session timeout (in
minutes). Note that extending the Keystone token
expiration may cause scalability issues in large and very busy SUSE OpenStack Cloud
installations.
Specify a regular expression with which to check the password. The
default expression (.{8,}
) tests for a minimum length
of 8 characters. The string you enter is interpreted as a Python regular
expression (see
http://docs.python.org/2.7/library/re.html#module-re
for a reference).
Error message that will be displayed in case the password validation fails.
Choose whether to encrypt public communication (
) or not ( ). If choosing , you have two choices. You can either or provide the locations for the certificate key pair files and,—optionally— the certificate chain file. Using self-signed certificates is for testing purposes only and should never be used in production environments!Making Horizon highly available requires no special configuration—it is sufficient to deploy it on a cluster.
Heat is a template-based orchestration engine that enables you to, for example, start workloads requiring multiple servers or to automatically restart instances if needed. It also brings auto-scaling to SUSE OpenStack Cloud by automatically starting additional instances if certain criteria are met. For more information about Heat refer to the OpenStack documentation at http://docs.openstack.org/developer/heat/.
Heat should be deployed on a Control Node. To make Heat highly available, deploy it on a cluster.
The following attributes can be configured for Heat:
Shows debugging output in the log files when set to
.Choose whether to encrypt public communication (SSL Support: Protocol for configuration details.
) or not ( ). If choosing , refer to
Heat uses Keystone Trusts to delegate a subset of user roles to the
Heat engine for deferred operations (see
Steve
Hardy's blog for details). It can either delegate all user roles or
only those specified in the trusts_delegated_roles
setting. Consequently, all roles listed in
trusts_delegated_roles
need to be assigned to a user,
otherwise the user will not be able to use Heat.
The recommended setting for trusts_delegated_roles
is
Member
, since this is the default role most users are
likely to have. This is also the default setting when installing SUSE OpenStack Cloud
from scratch.
On installations where this setting is introduced through an upgrade,
trusts_delegated_roles
will be set to
heat_stack_owner
. This is a conservative choice to
prevent breakage in situations where unprivileged users may already have
been assigned the heat_stack_owner
role to enable them
to use Heat but lack the Member
role. As long as you can
ensure that all users who have the heat_stack_owner
role
also have the Member
role, it is both safe and
recommended to change trusts_delegated_roles to Member
.
To view or change the trusts_delegated_role setting you need to open the
Heat barclamp and click trusts_delegated_roles
setting and modify the list of
roles as desired.
An empty value for trusts_delegated_roles
will delegate
all of user roles to Heat. This may create a security
risk for users who are assigned privileged roles, such as
admin
, because these privileged roles will also be
delegated to the Heat engine when these users create Heat stacks.
Making Heat highly available requires no special configuration—it is sufficient to deploy it on a cluster.
Ceilometer collects CPU and networking data from SUSE OpenStack Cloud. This data can be used by a billing system to enable customer billing. Deploying Ceilometer is optional.
For more information about Ceilometer refer to the OpenStack documentation at http://docs.openstack.org/developer/ceilometer/.
As of SUSE OpenStack Cloud Crowbar 8 data measuring is only supported for KVM, Xen and Windows instances. Other hypervisors and SUSE OpenStack Cloud features such as object or block storage will not be measured.
The following attributes can be configured for Ceilometer:
Specify an interval in seconds after which Ceilometer performs an update of the specified meter.
Set the interval after which to check whether to raise an alarm because a threshold has been exceeded. For performance reasons, do not set a value lower than the default (60s).
Specify how long to keep the data. -1 means that samples are kept in the database forever.
Shows debugging output in the log files when set to
.With the default value Section 2.3, “SSL Encryption” for background information and Section 11.4.6, “Enabling SSL” for installation instructions. The following additional configuration options will become available when choosing :
enabled, public communication is not be encrypted. Choose to use SSL for encryption. See
When set to true
, self-signed certificates are
automatically generated and copied to the correct locations. This
setting is for testing purposes only and should never be used in
production environments!
Location of the certificate key pair files.
Set this option to true
when using self-signed
certificates to disable certificate checks. This setting is for
testing purposes only and should never be used in production
environments!
Specify the absolute path to the CA certificate. This field is
mandatory, and leaving it blank will cause the barclamp to fail. To fix
this issue, you have to provide the absolute path to the CA
certificate, restart the apache2
service, and
re-deploy the barclamp.
When the certificate is not already trusted by the pre-installed list of trusted root certificate authorities, you need to provide a certificate bundle that includes the root and all intermediate CAs.
The Ceilometer component consists of five different roles:
The Ceilometer API server role. This role needs to be deployed on a Control Node. Ceilometer collects approximately 200 bytes of data per hour and instance. Unless you have a very huge number of instances, there is no need to install it on a dedicated node.
The polling agent listens to the message bus to collect data. It needs to be deployed on a Control Node. It can be deployed on the same node as
.The compute agents collect data from the compute nodes. They need to be deployed on all KVM and Xen compute nodes in your cloud (other hypervisors are currently not supported).
An agent collecting data from the Swift nodes. This role needs to be deployed on the same node as swift-proxy.
Making Ceilometer highly available requires no special configuration—it is sufficient to deploy the roles
and on a cluster. If you are using MySQL or PostgreSQL, you can use two nodes.Manila provides coordinated access to shared or distributed file systems, similar to what Cinder does for block storage. These file systems can be shared between instances in SUSE OpenStack Cloud.
Manila uses different back-ends. As of SUSE OpenStack Cloud Crowbar 8 currently supported back-ends include , , and . Two more back-end options, and are available for testing purposes and are not supported.
Manila uses some CephFS features that are currently not supported by the SUSE Linux Enterprise Server 12 SP3 CephFS kernel client:
RADOS namespaces
MDS path restrictions
Quotas
As a result, to access CephFS shares provisioned by Manila, you must use ceph-fuse. For details, see http://docs.openstack.org/developer/manila/devref/cephfs_native_driver.html.
When first opening the Manila barclamp, the default proposal
is already available for configuration. To replace it, first delete it by clicking the trashcan icon and then choose a different back-end in the section . Select a and—optionally—provide a . Activate the back-end with . Note that at least one back-end must be configured.The attributes that can be set to configure Cinder depend on the back-end:
The generic driver is included as a technology preview and is not supported.
Provide the name of the Enterprise Virtual Server that the selected back-end is assigned to.
IP address for mounting shares.
Provide a file-system name for creating shares.
IP address of the HNAS management interface for communication between Manila controller and HNAS.
HNAS username Base64 String required to perform tasks like creating file-systems and network interfaces.
HNAS user password. Required only if private key is not provided.
RSA/DSA private key necessary for connecting to HNAS. Required only if password is not provided.
Time in seconds to wait before aborting stalled HNAS jobs.
Host name of the Virtual Storage Server.
The name or IP address for the storage controller or the cluster.
The port to use for communication. Port 80 is usually used for HTTP, 443 for HTTPS.
Login credentials.
Transport protocol for communicating with the storage controller or cluster. Supported protocols are HTTP and HTTPS. Choose the protocol your NetApp is licensed for.
Set to true
to use Ceph deployed with Crowbar.
Lets you manually pick and configure a driver. Only use this option for testing purposes, it is not supported.
The Manila component consists of two different roles:
The Manila server provides the scheduler and the API. Installing it on a Control Node is recommended.
The shared storage service. It can be installed on a Control Node, but it is recommended to deploy it on one or more dedicated nodes supplied with sufficient disk space and networking capacity, since it will generate a lot of network traffic.
While the
role can be deployed on a cluster, deploying on a cluster is not supported. Therefore it is generally recommended to deploy on several nodes—this ensures the service continues to be available even when a node fails.Tempest is an integration test suite for SUSE OpenStack Cloud written in Python. It contains multiple integration tests for validating your SUSE OpenStack Cloud deployment. For more information about Tempest refer to the OpenStack documentation at http://docs.openstack.org/developer/tempest/.
Tempest is only included as a technology preview and not supported.
Tempest may be used for testing whether the intended setup will run without problems. It should not be used in a production environment.
Tempest should be deployed on a Control Node.
The following attributes can be configured for Tempest:
Credentials for a regular user. If the user does not exist, it will be created.
Tenant to be used by Tempest. If it does not exist, it will be created. It is safe to stick with the default value.
Credentials for an admin user. If the user does not exist, it will be created.
To run tests with Tempest, log in to the Control Node on which
Tempest was deployed. Change into the directory
/var/lib/openstack-tempest-test
. To get an overview of
available commands, run:
./tempest --help
To serially invoke a subset of all tests (“the gating
smoketests”) to help validate the working functionality of your
local cloud instance, run the following command. It will save the output to
a log file
tempest_CURRENT_DATE.log
.
./tempest run --smoke --serial 2>&1 \ | tee "tempest_$(date +%Y-%m-%d_%H%M%S).log"
Tempest cannot be made highly available.
Magnum is an OpenStack project which offers container orchestration engines for deploying and managing containers as first class resources in OpenStack.
For more information about Magnum, see the OpenStack documentation at http://docs.openstack.org/developer/magnum/.
For information on how to deploy a Kubernetes cluster (either from command line or from the Horizon Dashboard), see the Supplement to Administrator Guide and End User Guide. It is available from https://documentation.suse.com/soc/8/.
The following
can be configured for Magnum:
Deploying Kubernetes clusters in a cloud without an Internet connection
(see also https://documentation.suse.com/soc/8/single-html/suse-openstack-cloud-supplement/#sec-deploy-kubernetes-without)
requires the registry_enabled
option in its cluster
template set to true
. To make this offline scenario
work, you also need to set the option to true
. This restores the old,
insecure behavior for clusters with the
registry-enabled
or volume_driver=Rexray
options enabled.
Domain name to use for creating trustee for bays.
Increases the amount of information that is written to the log files when set to
.Shows debugging output in the log files when set to
.To store certificates, either use the OpenStack service, a local directory ( ), or the .
If you choose to use Barbican for managing certificates, make sure that the Barbican barclamp is enabled.
The Magnum barclamp consists of the following roles: Section 12.17.1, “HA Setup for Magnum”. When deploying the role onto a Control Node, additional RAM is required for the Magnum server. It is recommended to only deploy the role to a Control Node that has 16 GB RAM.
. It can either be deployed on a Control Node or on a cluster—seeMaking Magnum highly available requires no special configuration. It is sufficient to deploy it on a cluster.
Barbican is a component designed for storing secrets in a secure and standardized manner protected by Keystone authentication. Secrets include SSL certificates and passwords used by various OpenStack components.
Barbican settings can be configured in Raw
mode
only. To do this, open the Barbican barclamp configuration in mode.
When configuring Barbican, pay particular attention to the following settings:
bind_host
Bind host for the Barbican API service
bind_port
Bind port for the Barbican API service
processes
Number of API processes to run in Apache
ssl
Enable or disable SSL
threads
Number of API worker threads
debug
Enable or disable debug logging
enable_keystone_listener
Enable or disable the
Keystone listener services
kek
An encryption key (fixed-length 32-byte
Base64-encoded value) for Barbican's
simple_crypto
plug-in. If left unspecified, the
key will be generated automatically.
If you plan to restore and use the existing Barbican database after a full reinstall (including a complete wipe of the Crowbar node), make sure to save the specified encryption key beforehand. You will need to provide it after the full reinstall in order to access the data in the restored Barbican database.
With the default value Section 2.3, “SSL Encryption” for background information and Section 11.4.6, “Enabling SSL” for installation instructions. The following additional configuration options will become available when choosing :
, public communication will not be encrypted. Choose to use SSL for encryption. See
When set to true
, self-signed certificates are
automatically generated and copied to the correct locations. This
setting is for testing purposes only and should never be used in
production environments!
Location of the certificate key pair files.
Set this option to true
when using self-signed
certificates to disable certificate checks. This setting is for
testing purposes only and should never be used in production
environments!
Specify the absolute path to the CA certificate. This field is
mandatory, and leaving it blank will cause the barclamp to fail. To fix
this issue, you have to provide the absolute path to the CA
certificate, restart the apache2
service, and
re-deploy the barclamp.
When the certificate is not already trusted by the pre-installed list of trusted root certificate authorities, you need to provide a certificate bundle that includes the root and all intermediate CAs.
To make Barbican highly available, assign the
role to the Controller Cluster.Sahara provides users with simple means to provision data processing frameworks (such as Hadoop, Spark, and Storm) on OpenStack. This is accomplished by specifying configuration parameters such as the framework version, cluster topology, node hardware details, etc.
Set to true
to increase the amount of
information written to the log files.
Making Sahara highly available requires no special configuration. It is sufficient to deploy it on a cluster.
Ironic is the OpenStack bare metal service for provisioning physical machines. Refer to the OpenStack developer and admin manual for information on drivers, and administering Ironic.
Deploying the Ironic barclamp is done in five steps:
Set options in the Custom view of the barclamp.
List the enabled_drivers
in the Raw view.
Configure the Ironic network in network.json
.
Apply the barclamp to a Control Node.
Apply the
role to the same node you applied the Ironic barclamp to, in place of the other roles.Currently, there are two options in the Custom view of the barclamp.
Node cleaning prepares the node to accept a new workload. When you set this to
, Ironic collects a list of cleaning steps from the Power, Deploy, Management, and RAID interfaces of the driver assigned to the node. Ironic automatically prioritizes and executes the cleaning steps, and changes the state of the node to "cleaning". When cleaning is complete the state becomes "available". After a new workload is assigned to the machine its state changes to "active". disables automatic cleaning, and you must configure and apply node cleaning manually. This requires the admin to create and prioritize the cleaning steps, and to set up a cleaning network. Apply manual cleaning when you have long-running or destructive tasks that you wish to monitor and control more closely. (SeeSSL support is not yet enabled, so the only option is
.
You must enter the Raw view of barclamp and specify a list of drivers to load during service initialization.
pxe_ipmitool
is the recommended default Ironic driver. It uses the
Intelligent Platform Management Interface (IPMI) to control the power state
of your bare metal machines, creates the appropriate PXE configurations
to start them, and then performs the steps to provision and configure the machines.
"enabled_drivers": ["pxe_ipmitool"],
See Ironic Drivers for more information.
This is a complete Ironic network.json
example, using
the default network.json
, followed by a diff that shows
the Ironic-specific configurations.
{ "start_up_delay": 30, "enable_rx_offloading": true, "enable_tx_offloading": true, "mode": "single", "teaming": { "mode": 1 }, "interface_map": [ { "bus_order": [ "0000:00/0000:00:01", "0000:00/0000:00:03" ], "pattern": "PowerEdge R610" }, { "bus_order": [ "0000:00/0000:00:01.1/0000:01:00.0", "0000:00/0000:00:01.1/0000.01:00.1", "0000:00/0000:00:01.0/0000:02:00.0", "0000:00/0000:00:01.0/0000:02:00.1" ], "pattern": "PowerEdge R620" }, { "bus_order": [ "0000:00/0000:00:01", "0000:00/0000:00:03" ], "pattern": "PowerEdge R710" }, { "bus_order": [ "0000:00/0000:00:04", "0000:00/0000:00:02" ], "pattern": "PowerEdge C6145" }, { "bus_order": [ "0000:00/0000:00:03.0/0000:01:00.0", "0000:00/0000:00:03.0/0000:01:00.1", "0000:00/0000:00:1c.4/0000:06:00.0", "0000:00/0000:00:1c.4/0000:06:00.1" ], "pattern": "PowerEdge R730xd" }, { "bus_order": [ "0000:00/0000:00:1c", "0000:00/0000:00:07", "0000:00/0000:00:09", "0000:00/0000:00:01" ], "pattern": "PowerEdge C2100" }, { "bus_order": [ "0000:00/0000:00:01", "0000:00/0000:00:03", "0000:00/0000:00:07" ], "pattern": "C6100" }, { "bus_order": [ "0000:00/0000:00:01", "0000:00/0000:00:02" ], "pattern": "product" } ], "conduit_map": [ { "conduit_list": { "intf0": { "if_list": [ "1g1", "1g2" ] }, "intf1": { "if_list": [ "1g1", "1g2" ] }, "intf2": { "if_list": [ "1g1", "1g2" ] }, "intf3": { "if_list": [ "1g1", "1g2" ] } }, "pattern": "team/.*/.*" }, { "conduit_list": { "intf0": { "if_list": [ "?1g1" ] }, "intf1": { "if_list": [ "?1g2" ] }, "intf2": { "if_list": [ "?1g1" ] }, "intf3": { "if_list": [ "?1g2" ] } }, "pattern": "dual/.*/.*" }, { "conduit_list": { "intf0": { "if_list": [ "?1g1" ] }, "intf1": { "if_list": [ "?1g1" ] }, "intf2": { "if_list": [ "?1g1" ] }, "intf3": { "if_list": [ "?1g2" ] } }, "pattern": "single/.*/.*ironic.*" }, { "conduit_list": { "intf0": { "if_list": [ "?1g1" ] }, "intf1": { "if_list": [ "?1g1" ] }, "intf2": { "if_list": [ "?1g1" ] }, "intf3": { "if_list": [ "?1g1" ] } }, "pattern": "single/.*/.*" }, { "conduit_list": { "intf0": { "if_list": [ "?1g1" ] }, "intf1": { "if_list": [ "1g1" ] }, "intf2": { "if_list": [ "1g1" ] }, "intf3": { "if_list": [ "1g1" ] } }, "pattern": ".*/.*/.*" }, { "conduit_list": { "intf0": { "if_list": [ "1g1" ] }, "intf1": { "if_list": [ "?1g1" ] }, "intf2": { "if_list": [ "?1g1" ] }, "intf3": { "if_list": [ "?1g1" ] } }, "pattern": "mode/1g_adpt_count/role" } ], "networks": { "ironic": { "conduit": "intf3", "vlan": 100, "use_vlan": false, "add_bridge": false, "add_ovs_bridge": false, "bridge_name": "br-ironic", "subnet": "192.168.128.0", "netmask": "255.255.255.0", "broadcast": "192.168.128.255", "router": "192.168.128.1", "router_pref": 50, "ranges": { "admin": { "start": "192.168.128.10", "end": "192.168.128.11" }, "dhcp": { "start": "192.168.128.21", "end": "192.168.128.254" } }, "mtu": 1500 }, "storage": { "conduit": "intf1", "vlan": 200, "use_vlan": true, "add_bridge": false, "mtu": 1500, "subnet": "192.168.125.0", "netmask": "255.255.255.0", "broadcast": "192.168.125.255", "ranges": { "host": { "start": "192.168.125.10", "end": "192.168.125.239" } } }, "public": { "conduit": "intf1", "vlan": 300, "use_vlan": true, "add_bridge": false, "subnet": "192.168.122.0", "netmask": "255.255.255.0", "broadcast": "192.168.122.255", "router": "192.168.122.1", "router_pref": 5, "ranges": { "host": { "start": "192.168.122.2", "end": "192.168.122.127" } }, "mtu": 1500 }, "nova_fixed": { "conduit": "intf1", "vlan": 500, "use_vlan": true, "add_bridge": false, "add_ovs_bridge": false, "bridge_name": "br-fixed", "subnet": "192.168.123.0", "netmask": "255.255.255.0", "broadcast": "192.168.123.255", "router": "192.168.123.1", "router_pref": 20, "ranges": { "dhcp": { "start": "192.168.123.1", "end": "192.168.123.254" } }, "mtu": 1500 }, "nova_floating": { "conduit": "intf1", "vlan": 300, "use_vlan": true, "add_bridge": false, "add_ovs_bridge": false, "bridge_name": "br-public", "subnet": "192.168.122.128", "netmask": "255.255.255.128", "broadcast": "192.168.122.255", "ranges": { "host": { "start": "192.168.122.129", "end": "192.168.122.254" } }, "mtu": 1500 }, "bmc": { "conduit": "bmc", "vlan": 100, "use_vlan": false, "add_bridge": false, "subnet": "192.168.124.0", "netmask": "255.255.255.0", "broadcast": "192.168.124.255", "ranges": { "host": { "start": "192.168.124.162", "end": "192.168.124.240" } }, "router": "192.168.124.1" }, "bmc_vlan": { "conduit": "intf2", "vlan": 100, "use_vlan": true, "add_bridge": false, "subnet": "192.168.124.0", "netmask": "255.255.255.0", "broadcast": "192.168.124.255", "ranges": { "host": { "start": "192.168.124.161", "end": "192.168.124.161" } } }, "os_sdn": { "conduit": "intf1", "vlan": 400, "use_vlan": true, "add_bridge": false, "mtu": 1500, "subnet": "192.168.130.0", "netmask": "255.255.255.0", "broadcast": "192.168.130.255", "ranges": { "host": { "start": "192.168.130.10", "end": "192.168.130.254" } } }, "admin": { "conduit": "intf0", "vlan": 100, "use_vlan": false, "add_bridge": false, "mtu": 1500, "subnet": "192.168.124.0", "netmask": "255.255.255.0", "broadcast": "192.168.124.255", "router": "192.168.124.1", "router_pref": 10, "ranges": { "admin": { "start": "192.168.124.10", "end": "192.168.124.11" }, "dhcp": { "start": "192.168.124.21", "end": "192.168.124.80" }, "host": { "start": "192.168.124.81", "end": "192.168.124.160" }, "switch": { "start": "192.168.124.241", "end": "192.168.124.250" } } } } }
This diff should help you separate the Ironic items from the default
network.json
.
--- network.json 2017-06-07 09:22:38.614557114 +0200 +++ ironic_network.json 2017-06-05 12:01:15.927028019 +0200 @@ -91,6 +91,12 @@ "1g1", "1g2" ] + }, + "intf3": { + "if_list": [ + "1g1", + "1g2" + ] } }, "pattern": "team/.*/.*" @@ -111,6 +117,11 @@ "if_list": [ "?1g1" ] + }, + "intf3": { + "if_list": [ + "?1g2" + ] } }, "pattern": "dual/.*/.*" @@ -131,6 +142,36 @@ "if_list": [ "?1g1" ] + }, + "intf3": { + "if_list": [ + "?1g2" + ] + } + }, + "pattern": "single/.*/.*ironic.*" + }, + { + "conduit_list": { + "intf0": { + "if_list": [ + "?1g1" + ] + }, + "intf1": { + "if_list": [ + "?1g1" + ] + }, + "intf2": { + "if_list": [ + "?1g1" + ] + }, + "intf3": { + "if_list": [ + "?1g1" + ] } }, "pattern": "single/.*/.*" @@ -151,6 +192,11 @@ "if_list": [ "1g1" ] + }, + "intf3": { + "if_list": [ + "1g1" + ] } }, "pattern": ".*/.*/.*" @@ -171,12 +217,41 @@ "if_list": [ "?1g1" ] + }, + "intf3": { + "if_list": [ + "?1g1" + ] } }, "pattern": "mode/1g_adpt_count/role" } ], "networks": { + "ironic": { + "conduit": "intf3", + "vlan": 100, + "use_vlan": false, + "add_bridge": false, + "add_ovs_bridge": false, + "bridge_name": "br-ironic", + "subnet": "192.168.128.0", + "netmask": "255.255.255.0", + "broadcast": "192.168.128.255", + "router": "192.168.128.1", + "router_pref": 50, + "ranges": { + "admin": { + "start": "192.168.128.10", + "end": "192.168.128.11" + }, + "dhcp": { + "start": "192.168.128.21", + "end": "192.168.128.254" + } + }, + "mtu": 1500 + }, "storage": { "conduit": "intf1", "vlan": 200,
With a successful deployment of the OpenStack Dashboard, the SUSE OpenStack Cloud Crowbar installation is finished. To be able to test your setup by starting an instance one last step remains to be done—uploading an image to the Glance component. Refer to the Supplement to Administrator Guide and End User Guide, chapter Manage images for instructions.
Now you can hand over to the cloud administrator to set up users, roles,
flavors, etc.—refer to the Administrator Guide for details. The default
credentials for the OpenStack Dashboard are user name admin
and password crowbar
.
SUSE OpenStack Cloud Crowbar supports integration with SUSE Enterprise Storage (SES), enabling Ceph block storage as well as image storage services in SUSE OpenStack Cloud.
To enable SES integration on Crowbar, an SES configuration file must be
uploaded to Crowbar. SES integration functionality is included in the
crowbar-core
package and can be used with the Crowbar UI
or CLI (crowbarctl
). The SES configuration file
describes various aspects of the Ceph environment, and keyrings for each
user and pool created in the Ceph environment for SUSE OpenStack Cloud Crowbar services.
For SES deployments that are version 5.5 and higher, a Salt runner is used to create all the users and pools. It also generates a YAML configuration that is needed to integrate with SUSE OpenStack Cloud. The integration runner creates separate users for Cinder, Cinder backup (not used by Crowbar currently) and Glance. Both the Cinder and Nova services have the same user, because Cinder needs access to create objects that Nova uses.
Configure SES with the following steps:
Login as root
and run the SES 5.5 Salt runner on the
Salt admin host.
root #
salt-run --out=yaml openstack.integrate prefix=mycloud
The prefix parameter allows pools to be created with the specified prefix. By using different prefix parameters, multiple cloud deployments can support different users and pools on the same SES deployment.
A YAML file is created with content similar to the following example:
ceph_conf: cluster_network: 10.84.56.0/21 fsid: d5d7c7cb-5858-3218-a36f-d028df7b0673 mon_host: 10.84.56.8, 10.84.56.9, 10.84.56.7 mon_initial_members: ses-osd1, ses-osd2, ses-osd3 public_network: 10.84.56.0/21 cinder: key: ABCDEFGaxefEMxAAW4zp2My/5HjoST2Y87654321== rbd_store_pool: mycloud-cinder rbd_store_user: cinder cinder-backup: key: AQBb8hdbrY2bNRAAqJC2ZzR5Q4yrionh7V5PkQ== rbd_store_pool: mycloud-backups rbd_store_user: cinder-backup glance: key: AQD9eYRachg1NxAAiT6Hw/xYDA1vwSWLItLpgA== rbd_store_pool: mycloud-glance rbd_store_user: glance nova: rbd_store_pool: mycloud-nova radosgw_urls: - http://10.84.56.7:80/swift/v1 - http://10.84.56.8:80/swift/v1
Upload the generated YAML file to Crowbar using the UI or
crowbarctl
CLI.
If the Salt runner is not available, you must manually create pools and users to allow SUSE OpenStack Cloud services to use the SES/Ceph cluster. Pools and users must be created for Cinder, Nova, and Glance. Instructions for creating and managing pools, users and keyrings can be found in the SUSE Enterprise Storage Administration Guide in the Key Management section.
After the required pools and users are set up on the SUSE Enterprise Storage/Ceph
cluster, create an SES configuration file in YAML format (using the
example template above). Upload this file to Crowbar using the UI or
crowbarctl
CLI.
As indicated above, the SES configuration file can be uploaded to Crowbar
using the UI or crowbarctl
CLI.
From the main Crowbar UI, the upload page is under
› .
If a configuration is already stored in Crowbar, it will be visible in
the upload page. A newly uploaded configuration will replace existing
one. The new configuration will be applied to the cloud on the next
chef-client
run. There is no need to reapply
proposals.
Configurations can also be deleted from Crowbar. After deleting a configuration, you must manually update and reapply all proposals that used SES integration.
With the crowbarctl
CLI, the command crowbarctl ses
upload FILE
accepts a path to the
SES configuration file.
SES integration with SUSE OpenStack Cloud services is implemented with relevant Barclamps
and installed with the crowbar-openstack
package.
Set Use SES Configuration
to true
under RADOS Store Parameters
. The Glance barclamp
pulls the uploaded SES configuration from Crowbar when applying the
Glance proposal and on chef-client
runs. If the SES
configuration is uploaded before the Glance proposal is created,
Use SES Configuration
is enabled automatically
upon proposal creation.
Create a new RADOS backend and set Use SES
Configuration
to true
. The Cinder
barclamp pulls the uploaded SES configuration from Crowbar when applying the
Cinder proposal and on chef-client
runs. If
the SES configuration was uploaded before the Cinder proposal
was created, a ses-ceph
RADOS backend is created
automatically on proposal creation with Use SES
Configuration
already enabled.
To connect with volumes stores in SES, Nova uses the configuration
from the Cinder barclamp.
For ephemeral storage, Nova re-uses the rbd_store_user
and key
from Cinder but has a separate rbd_store_pool
defined in the SES configuration. Ephemeral storage on SES can be
enabled or disabled by setting Use Ceph RBD Ephemeral Backend
in Nova proposal. In new deployments it is enabled by default.
In existing ones it is disabled for compatibility reasons.
Besides block storage, the SES cluster can also be used as a Swift
replacement for object storage. If radosgw_urls
section is present
in uploaded SES configuration, first of the URLs is registered
in the Keystone catalog as the "Swift"/"object-store" service. Some
configuration is needed on SES side to fully integrate with Keystone
auth.
If SES integration is enabled on a cloud with Swift deployed,
SES object storage service will get higher priority by default. To
override this and use Swift for object storage instead, remove
radosgw_urls
section from the SES configuration file and re-upload it
to Crowbar. Re-apply Swift proposal or wait for next periodic
chef-client run to make changes effective.
The following table lists all roles (as defined in the barclamps), and their
associated services. As of SUSE OpenStack Cloud Crowbar 8 this list is work
in progress. Services can be manually started and stopped with the commands
systemctl start SERVICE
and
systemctl stop SERVICE
.
Role |
Service |
---|---|
ceilometer-agent |
openstack-ceilometer-agent-compute
|
ceilometer-polling ceilometer-server ceilometer-swift-proxy-middleware |
|
| |
| |
| |
| |
| |
cinder-controller |
|
| |
cinder-volume |
|
database-server |
|
glance-server |
|
| |
heat-server |
|
| |
| |
| |
horizon |
|
keystone-server |
|
manila-server |
|
| |
manila-share |
|
neutron-server |
|
nova-compute-* |
|
| |
nova-controller |
|
| |
| |
| |
| |
| |
| |
rabbitmq-server |
|
swift-dispersion |
none |
swift-proxy |
|
swift-ring-compute |
none |
swift-storage |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
|
This is the documentation for the crowbar batch
subcommand.
crowbar batch
provides a quick way of creating, updating,
and applying Crowbar proposals. It can be used to:
Accurately capture the configuration of an existing Crowbar environment.
Drive Crowbar to build a complete new environment from scratch.
Capture one SUSE OpenStack Cloud environment and then reproduce it on another set of hardware (provided hardware and network configuration match to an appropriate extent).
Automatically update existing proposals.
As the name suggests, crowbar batch
is intended to be run
in “batch mode” that is mostly unattended. It has two modes of
operation:
Exports a YAML file which describes existing proposals and how their parameters deviate from the default proposal values for that barclamp.
Imports a YAML file in the same format as above. Uses it to build new proposals if they do not yet exist. Updates the existing proposals so that their parameters match those given in the YAML file.
Here is an example YAML file. At the top-level there is a proposals array, each entry of which is a hash representing a proposal:
proposals: - barclamp: provisioner # Proposal name defaults to 'default'. attributes: shell_prompt: USER@ALIAS:CWD SUFFIX - barclamp: database # Default attributes are good enough, so we just need to assign # nodes to roles: deployment: elements: database-server: - "@@controller1@@" - barclamp: rabbitmq deployment: elements: rabbitmq-server: - "@@controller1@@"
Note that the characters @
and `
are
reserved indicators in YAML. They can appear anywhere in a string
except at the beginning. Therefore a string such as
@@controller1@@
needs to be quoted using double quotes.
Name of the barclamp for this proposal (required).
Name of this proposal (optional; default is default
).
In build
mode, if the proposal does not already
exist, it will be created.
An optional nested hash containing any attributes for this proposal which deviate from the defaults for the barclamp.
In export
mode, any attributes set to the default
values are excluded to keep the YAML as short and readable as possible.
In build
mode, these attributes are deep-merged with
the current values for the proposal. If the proposal did not already
exist, batch build will create it first. The attributes are merged with
the default values for the barclamp's proposal.
An optional array of paths to nested attributes which should be removed from the proposal.
Each path is a period-delimited sequence of attributes; for example
pacemaker.stonith.sbd.nodes
would remove all SBD
nodes from the proposal if it already exists. If a path segment contains
a period, it should be escaped with a backslash, for example
segment-one.segment\.two.segment_three
.
This removal occurs before the deep merge described above.
For example, think of a YAML file which includes a Pacemaker barclamp
proposal where the wipe_attributes
entry contains
pacemaker.stonith.sbd.nodes
. A batch build with this
YAML file ensures that only SBD nodes listed in the attributes
sibling
hash are used at the end of the run. In contrast,
without the wipe_attributes
entry, the given SBD
nodes would be appended to any SBD nodes already defined in the
proposal.
A nested hash defining how and where this proposal should be deployed.
In build
mode, this hash is deep-merged in the same
way as the attributes hash, except that the array of elements for each
Chef role is reset to the empty list before the deep merge. This
behavior may change in the future.
A string like @@node@@
(where
node is a node alias) will be substituted for
the name of that node, no matter where the string appears in the YAML file.
For example, if controller1
is a Crowbar alias for node
d52-54-02-77-77-02.mycloud.com
, then
@@controller1@@
will be substituted for that host name.
This allows YAML files to be reused across environments.
In addition to the standard options available to every
crowbar
subcommand (run crowbar batch
--help
for a full list), there are some extra options
specifically for crowbar batch
:
Only include the barclamp / proposals given.
This option can be repeated multiple times. The inclusion value can
either be the name of a barclamp (for example,
pacemaker
) or a specifically named proposal within
the barclamp (for example, pacemaker.network_cluster
).
If it is specified, then only the barclamp / proposals specified are included in the build or export operation, and all others are ignored.
This option can be repeated multiple times. The exclusion value is the
same format as for --include
. The barclamps / proposals
specified are excluded from the build or export operation.
Change the timeout for Crowbar API calls.
As Chef's run lists grow, some of the later OpenStack barclamp proposals (for example Nova, Horizon, or Heat) can take over 5 or even 10 minutes to apply. Therefore you may need to increase this timeout to 900 seconds in some circumstances.
To limit users' access rights (or to define more fine-grained access
rights), you can use Role Based Access Control (RBAC, only available with
Keystone v3). In the example below, we will create a
new role (ProjectAdmin
). It allows users with this role to
add and remove other users to the Member
role on the same
project.
To create a new role that can be assigned to a user-project pair, the following basic steps are needed:
Create a custom policy.json
file for the
Keystone component. On the node where the keystone-server
role is
deployed, copy the file to
/etc/keystone/CUSTOM_policy.json
.
For details, see Section 13.1, “Editing policy.json
”.
Create a custom keystone_policy.json
file for the
Horizon component. On the node where the
nova_dashboard-server
role is deployed, copy the custom
keystone_policy.json
file to
/srv/www/openstack-dashboard/openstack_dashboard/conf/
(default directory for policy files in Horizon). For details, see
Section 13.2, “Editing keystone_policy.json
”.
Make the Keystone component aware of the
CUSTOM_policy.json
file by
editing and reapplying the barclamp. For details,
see Section 13.3, “Adjusting the . Barclamp
Proposal”
Make the Horizon component aware of the
keystone_policy.json
file by editing and reapplying
the barclamp. For details, see
Section 13.4, “Adjusting the . Barclamp
Proposal”
policy.json
# The policy.json
file is located in
/etc/keystone/
on the node where the
keystone-server
role is deployed.
Copy /etc/keystone/policy.json
and save it under
a different name, for example
CUSTOM_policy.json
.
If you use the same name as the original file, your custom file will be overwritten by the next package update.
To add the new role, enter the following two lines at the beginning of the file:
{ "subadmin": "role:ProjectAdmin", "projectadmin": "rule:subadmin and project_id:%(target.project.id)s", [...]
Adjust the other rules in the file accordingly:
"identity:get_domain": "rule:admin_required or rule:subadmin", [...] "identity:get_project": "rule:admin_required or rule:projectadmin", [...] "identity:list_user_projects": "rule:admin_or_owner or rule:projectadmin", [...] "identity:update_project": "rule:admin_required or rule:projectadmin", [...] "identity:get_user": "rule:admin_required or rule:projectadmin", "identity:list_users": "rule:admin_required or rule:subadmin", [...] "identity:list_groups": "rule:admin_required or rule:subadmin", [...] "identity:list_roles": "rule:admin_required or rule:subadmin", [...] "identity:list_grants": "rule:admin_required or (rule:subadmin and project_id:%(target.project.id)s)", "identity:create_grant": "rule:admin_required or (rule:subadmin and project_id:%(target.project.id)s and 'Member':%(target.role.name)s)", "identity:revoke_grant": "rule:admin_required or (rule:subadmin and project_id:%(target.project.id)s and 'Member':%(target.role.name)s)", [...] "identity:list_role_assignments": "rule:admin_required or rule:subadmin",
Save the changes.
On the node where the keystone-server
role is
deployed, copy the file to
/etc/keystone/CUSTOM_policy.json
.
Usually, the keystone-server
role is deployed to a Control Node
(or to a cluster, if you use a High Availability setup).
keystone_policy.json
#By default, the keystone_policy.json
file is
located in
/srv/www/openstack-dashboard/openstack_dashboard/conf/
on the node where the nova_dashboard-server
role is
deployed. It is similar (but not identical) to
policy.json
and defines which actions the user with a
certain role is allowed to execute in Horizon. If the user is not
allowed to execute a certain action, the OpenStack Dashboard will show an
error message.
Copy
/srv/www/openstack-dashboard/openstack_dashboard/conf/keystone_policy.json
and save it under a different name, for example
CUSTOM_keystone_policy.json
.
If you use the same name as the original file, your custom file will be overwritten by the next package update.
To add the new role, enter the following two lines at the beginning of the file:
{ "subadmin": "role:ProjectAdmin", "projectadmin": "rule:subadmin and project_id:%(target.project.id)s", [...]
Adjust the other rules in the file accordingly:
"identity:get_project": "rule:admin_required or rule:projectadmin", [...] "identity:list_user_projects": "rule:admin_or_owner or rule:projectadmin", [...] "identity:get_user": "rule:admin_required or rule:projectadmin", "identity:list_users": "rule:admin_required or rule:subadmin", [...] "identity:list_roles": "rule:admin_required or rule:subadmin", [...] "identity:list_role_assignments": "rule:admin_required or rule:subadmin",
Save the changes and copy the file to
/srv/www/openstack-dashboard/openstack_dashboard/conf/CUSTOM_keystone_policy.json
on the node where the nova_dashboard-server
role is
deployed.
Log in to the Crowbar Web interface.
Select
› .Go to the
barclamp and click .In the
section, click . This shows the complete configuration file and allows you to edit it directly.Adjust the policy_file
parameter to point to the
CUSTOM_policy.json
file.
For example:
{ [...] "policy_file": "mypolicy.json",
and the changes to the Keystone barclamp.
Log in to the Crowbar Web interface.
Select
› .Go to the
barclamp and click .In the
section, click . This shows the complete configuration file and allows you to edit it directly.If needed, adjust the policy_file_path
parameter to
point to the directory where you copied the newly added
CUSTOM_keystone_policy.json
file. By default, its value is an empty string—this means that
the default directory will be used.
Enter the new file's name as value of the identity
parameter within the policy_file
section (1):
{
"policy_file_path": "",
"policy_file": {
"identity": "mykeystone_policy.json", 1
"compute": "nova_policy.json",
"volume": "cinder_policy.json",
"image": "glance_policy.json",
"orchestration": "heat_policy.json",
"network": "neutron_policy.json",
"telemetry": "ceilometer_policy.json"
and the changes to the Horizon barclamp.
The following are the roles defined in SUSE OpenStack Cloud Crowbar. These roles serve as a way to group common administrative needs at the OpenStack service level. Each role represents administrative privilege into each service. Multiple roles can be assigned to a user. You can assign a Service Admin Role to a user once you have determined that the user is authorized to perform administrative actions and access resources in that service.
The main components of Service Administrator Roles are:
nova_admin
role in the identity service (Keystone) and
support in nova policy.json
. Assign this role to
users whose job function it is to perform Nova compute-related
administrative tasks.
neutron_admin
role in the identity service and support
in neutron policy.json
. Assign this role to users
whose job function it is to perform neutron networking-related
administrative tasks.
cinder_admin
role in the identity service and support
in cinder policy.json
. Assign this role to users
whose job function it is to perform Cinder storage-related administrative
tasks.
glance_admin
role in the identity service and support
in glance policy.json
. Assign this role to users
whose job function it is to perform Cinder storage-related administrative
tasks.
glance_policy.json
may Introduce a
Security Issue
The OpenStack Security Note OSSN-0075 https://wiki.openstack.org/wiki/OSSN/OSSN-0075 describes a scenario where a malicious tenant is able to reuse deleted Glance image IDs to share malicious images with other tenants in a manner that is undetectable to the victim tenant.
The default policy glance_policy.json
that is
shipped with SUSE OpenStack Cloud Crowbar prevents this by ensuring only admins can
deactivate/reactivate images:
"deactivate": "role:admin" "reactivate": "role:admin"
SUSE suggests these settings should not be changed. If you do change them please refer to the OSSN-0075 https://wiki.openstack.org/wiki/OSSN/OSSN-0075 for details on the exact scope of the security issue.
Typically, each OpenStack component comes with a configuration file, for
example: /etc/nova/nova.conf
.
These configuration files can still be used. However, to configure an
OpenStack component and its different components and roles, it is now
preferred to add custom configuration file snippets to a
SERVICE.conf.d/
directory
instead.
By default, a configuration snippet with a basic configuration for each OpenStack component is available in the following directory:
/etc/SERVICE/SERVICE.conf.d/010-SERVICE.conf
For example: /etc/nova/nova.conf.d/010-nova.conf
Those files should not be modified.
To adjust or overwrite settings for the respective OpenStack component, add a
custom configuration file to the same directory,
/etc/SERVICE/SERVICE.conf.d/
.
The same applies if you want to configure individual components or roles of
an OpenStack component, like nova-api
or
nova-compute
, for example. But in this case, add your
custom configuration file to the following directory:
/etc/SERVICE/ROLE.conf.d/
For example: /etc/nova/nova-api.conf.d/
All custom configuration file must follow the rules listed in Section 14.3, “Naming Conventions for Custom Configuration Files”.
Use the following rules for any configuration files you add:
The file name must start with a 3-digit number and a dash. For example:
/etc/nova/nova.conf.d/500-nova.conf
The file must have the following file name extension:
.conf
For configuration management systems (for example: Crowbar, Salt), use
numbers between 100
and 499
.
To override settings written by the configuration management system, use
numbers starting from 500
. They have higher priority.
The configuration files are processed in the following order:
/etc/SERVICE/SERVICE.conf
/etc/SERVICE/SERVICE.conf.d/*.conf
(in dictionary order)
/etc/SERVICE/ROLE.conf.d/*.conf
(in dictionary order)
If conflicting values are set for the same parameter, the last configured value overwrites all previous ones. In particular, values defined in
/etc/SERVICE/SERVICE.conf.d/XXX-SERVICE.conf
overwrite configuration values in
/etc/SERVICE/SERVICE.conf
For details, also see
/etc/SERVICE/README.config
.
This chapter describes how to install SUSE CaaS Platform Heat template on SUSE OpenStack Cloud Crowbar.
Download the latest SUSE CaaS Platform for OpenStack image (for example,
SUSE-CaaS-Platform-3.0-OpenStack-Cloud.x86_64-1.0.0-GM.qcow2
)
from https://download.suse.com.
Upload the image to Glance:
openstack image create --public --disk-format qcow2 --container-format \ bare --file SUSE-CaaS-Platform-3.0-OpenStack-Cloud.x86_64-1.0.0-GM.qcow2 \ CaaSP-3
Install the caasp-openstack-heat-templates package on a machine with SUSE OpenStack Cloud Crowbar repositories:
zypper in caasp-openstack-heat-templates
The installed templates are located in
/usr/share/caasp-openstack-heat-templates
.
Alternatively, you can get official Heat templates by cloning the appropriate Git repository:
git clone https://github.com/SUSE/caasp-openstack-heat-templates
In Horizon, go to
› › .
Select caasp-stack.yaml
file.
In the
dialog, provide the required information (stack name, password, flavor size, external network of your environment, etc.).Click OpenStack environment. The stack creates one Admin Node, one Master Node, and server worker nodes as specified.
to launch the stack. This creates all required resources for running SUSE CaaS Platform in an
Specify the appropriate flavor and network settings in the
caasp-environment.yaml
file.
Create a stack in Heat by passing the template, environment file, and parameters:
openstack stack create -t caasp-stack.yaml -e caasp-environment.yaml \ --parameter image=CaaSP-3 caasp-stack
After the stack has been created, the Velum SUSE CaaS Platform dashboard runs on the Admin Node. You can access it using the Admin Node's floating IP address.
Create an account and follow the steps in the Velum SUSE CaaS Platform dashboard to complete the SUSE CaaS Platform installation.
When you have successfully accessed the admin node web interface via the floating IP, follow the instructions at https://documentation.suse.com/suse-caasp/3/single-html/caasp-deployment/ to continue the setup of SUSE CaaS Platform.
A Heat stack with load balancing and multiple master nodes can only be created from the command line, because Horizon does not have support for nested Heat templates.
Install the caasp-openstack-heat-templates package on a machine with SUSE OpenStack Cloud Crowbar repositories:
zypper in caasp-openstack-heat-templates
The installed templates are located in
/usr/share/caasp-openstack-heat-templates
.
A working load balancer is needed in your SUSE OpenStack Cloud deployment. SUSE OpenStack Cloud Crowbar uses HAProxy.
Verify that load balancing with HAProxy is working correctly
in your OpenStack installation by creating a load balancer manually and
checking that the provisioning_status
changes to
Active
.
tux >
openstack loadbalancer show
<LOAD_BALANCER_ID>
HAProxy is the default load balancer provider in SUSE OpenStack Cloud Crowbar.
The steps below can be used to set up a network, subnet, router, security
and IPs for a test lb_net1
network with
lb_subnet1
subnet.
tux >
openstack network create lb_net1tux >
openstack subnet create --name lb_subnet1 lb_net1 \ --subnet-range 172.29.0.0/24 --gateway 172.29.0.2tux >
openstack router create lb_router1tux >
openstack router add subnet lb_router1 lb_subnet1tux >
openstack router set lb_router1 --external-gateway ext-nettux >
openstack network list
Specify the appropriate flavor and network settings in the
caasp-multi-master-environment.yaml
file.
Set master_count
to the desired number in the
caasp-multi-master-environment.yaml
file. The master
count must be set to an odd number of nodes.
master_count: 3
Create a stack in Heat by passing the template, environment file, and parameters:
tux >
openstack stack create -t caasp-multi-master-stack.yaml \
-e caasp-multi-master-environment.yaml --parameter image=CaaSP-3 caasp-multi-master-stack
Find the floating IP address of the load balancer. This is necessary for accessing the Velum SUSE CaaS Platform dashboard.
tux >
openstack loadbalancer list --provider
From the output, copy the id
and enter it in the
following command as shown in the following example:
tux >
openstack loadbalancer show id
+---------------------+------------------------------------------------+ | Field | Value | +---------------------+------------------------------------------------+ | admin_state_up | True | | description | | | id | 0d973d80-1c79-40a4-881b-42d111ee9625 | | listeners | {"id": "c9a34b63-a1c8-4a57-be22-75264769132d"} | | | {"id": "4fa2dae0-126b-4eb0-899f-b2b6f5aab461"} | | name | caasp-stack-master_lb-bhr66gtrx3ue | | operating_status | ONLINE | | pools | {"id": "8c011309-150c-4252-bb04-6550920e0059"} | | | {"id": "c5f55af7-0a25-4dfa-a088-79e548041929"} | | provider | haproxy | | provisioning_status | ACTIVE | | tenant_id | fd7ffc07400642b1b05dbef647deb4c1 | | vip_address | 172.28.0.6 | | vip_port_id | 53ad27ba-1ae0-4cd7-b798-c96b53373e8b | | vip_subnet_id | 87d18a53-ad0c-4d71-b82a-050c229b710a | +---------------------+------------------------------------------------+
Search the floating IP list for vip_address
tux >
openstack floating ip list | grep 172.28.0.6
| d636f3...481b0c | fd7ff...deb4c1 | 172.28.0.6 | 10.84.65.37 | 53ad2...373e8b |
The load balancer floating ip address is 10.84.65.37
Accessing the Velum SUSE CaaS Platform Dashboard
After the stack has been created, the Velum SUSE CaaS Platform dashboard runs on the admin node. You can access it using the floating IP address of the admin node.
Create an account and follow the steps in the Velum SUSE CaaS Platform dashboard to complete the SUSE CaaS Platform installation.
SUSE CaaS Platform Admin Node Install: Screen 1
SUSE CaaS Platform Admin Node Install: Screen 2
SUSE CaaS Platform Admin Node Install: Screen 3
SUSE CaaS Platform Admin Node Install: Screen 4
SUSE CaaS Platform Admin Node Install: Screen 5
Set External Kubernetes API to LOADBALANCER_FLOATING_IP, External Dashboard FQDN to ADMIN_NODE_FLOATING_IP
SUSE CaaS Platform Admin Node Install: Screen 6
SUSE CaaS Platform Admin Node Install: Screen 7
When deploying a CaaaSP cluster using SUSE CaaS Platform OpenStack Heat
templates, the following CPI parameters can be set in
caasp-environment.yaml
or
caasp-multi-master-environment.yaml
.
The URL of the Keystone API used to authenticate the user. This value can be found on OpenStack Dashboard under › › (for example, https://api.keystone.example.net:5000/v3/)
Name of the domain the user belongs to
Name of the project the user belongs to. This is the project where the resources will be created.
Name of the region to use when running a multi-region OpenStack cloud. The region is a general division of an OpenStack deployment.
Username of a valid user that has been set in Keystone. Default: admin
Password of a valid user that has been set in Keystone.
Neutron load balancer monitoring max retries. Default: 3
Cinder Block Storage API version. Possible values are v1, v2 , v3 or auto. Default: auto
Ignore Cinder and Nova availability zones. Default: true
When the SUSE CaaS Platform cluster has been deployed by the Heat templates, the
Velum dashboard on the admin node
(https://ADMIN-NODE-IP/) will have entries for
OpenStack
settings form will show the values that were set in the
caasp-environment.yaml
or
caasp-multi-master-environment.yaml
files.
After the SUSE CaaS Platform cluster comes up, install the latest SUSE CaaS Platform 3.0 Maintenance Update using the following steps:
Spin up a SUSE CaaS Platform cluster using Heat templates following the instructions in Section 15.1, “SUSE CaaS Platform Heat Installation Procedure”. Do not go through the bootstrapping steps, because the SUSE CaaS Platform Maintenance Update (MU) must be applied first. After the maintenance update process below succeeds, return to the SUSE CaaS Platform installation instructions and follow the admin node bootstrapping steps.
Apply the SUSE CaaS Platform 3.0 Maintenance Update with the following steps:
Log in to each node and add the update repository.
tux >
sudo zypper ar http://nu.novell.com/SUSE/Updates/SUSE-CAASP/3.0/x86_64/update/ caasp_update
With root privileges, run transactional-update
on each node.
Reboot each node
Verify that the Velum image packages were updated
tux >
zypper se --detail velum-image
i | sles12-velum-image | package | 3.1.7-3.27.3 | x86_64 | update_caasp
Upload a valid trust certificate that can validate a certificate
presented by Keystone at the specified
keystone_auth_url
in the system-wide
certificate
section of Velum. If the SSL certificate provided
by Keystone cannot be verified, bootstrapping fails with the error
message x509: certificate signed by unknown authority
.
If your OpenStack endpoints operate on the Internet, or if the SSL certificates in use have been signed by a public authority, no action is needed to enable secure communication with them.
If your OpenStack services operate in a private network using SSL certificates signed by an organizational certificate authority, provide that CA certificate as the system-wide certificate.
If your OpenStack service SSL infrastructure was self-signed during the
installation of SUSE OpenStack Cloud Crowbar (as is done by default), its CA
certificate (with the file extension .pem
) can be
retrieved from the admin node in the
/etc/ssl/certs/
directory. The filename should
match the node name of your primary controller node. Download this file
and provide it as the system-wide certificate.
After the nodes come up, continue with the installation instructions in Section 15.1, “SUSE CaaS Platform Heat Installation Procedure” following the admin node cluster bootstrapping steps.
More information about SUSE CaaS Platform is available at https://documentation.suse.com/suse-caasp/3/single-html/caasp-deployment/
In addition to OpenStack barclamps, SUSE OpenStack Cloud includes several components that can be configured using the appropriate Crowbar barclamps.
In addition to OpenStack barclamps, SUSE OpenStack Cloud includes several components that can be configured using the appropriate Crowbar barclamps.
Crowbar is a self-referential barclamp used for enabling other barclamps. By creating a Crowbar proposal, you can modify the default number of threads and workers. This way, you can scale the admin server according to the actual usage or the number of available cores of the admin node.
To change the default settings, create a Crowbar proposal and switch to the
workers
and threads
values. The number of threads should be set to the number of available
cores. The default number of workers should be increased to 3 if the
graphical interface becomes slow. Save and apply the changes using the
appropriate buttons.
The NTP service is responsible for keeping the clocks in your cloud servers in sync. Among other things, synchronized clocks ensure that the chef-client works properly. It also makes it easier to read logs from different nodes by correlating timestamps in them. The NTP component is deployed on the Administration Server automatically using the default settings. The NTP barclamp can be used to specify IP addresses of the external NTP servers and assign specific roles to the desired nodes. The following parameter can be configured using the NTP barclamp:
A comma-separated list of IP addresses of external NTP servers.
The NTP service consists of two different roles:
A node that acts as an NTP server for NTP clients in your cloud. There can be more than one node with the ntp-server role in your cloud. In this scenario, the NTP server nodes can communicate with each other and the specified external servers to keep their time in sync.
The ntp-client
role can be assigned to any
node. Nodes with the ntp-client role assigned to them keep their time in
sync using NTP servers in your cloud.
The purpose of this document is to help set up SSL Support for several services in SUSE OpenStack Cloud. The scope of this document covers all public endpoints in your OpenStack cluster. In most cases you want to have a Secure CA or External CA where your certificates are signed. You will sign with …
Find a list of log files below, sorted according to the nodes where they can be found.
Find solutions for the most common pitfalls and technical details on how to create a support request for SUSE OpenStack Cloud Crowbar here.
Keeping the nodes in SUSE OpenStack Cloud Crowbar up-to-date requires an appropriate setup of the update and pool repositories and the deployment of either the barclamp or the SUSE Manager barclamp. For details, see Section 5.2, “Update and Pool Repositories”, Section 11.4.1, “Deploying Node Updates with the Updater Barclamp”, and Section 11.4.2, “Configuring Node Updates with the . Barclamp”
If one of those barclamps is deployed, patches are installed on the nodes. Patches that do not require a reboot will not cause a service interruption. If a patch (for example, a kernel update) requires a reboot after the installation, services running on the machine that is rebooted will not be available within SUSE OpenStack Cloud. Therefore, we strongly recommend installing those patches during a maintenance window.
As of SUSE OpenStack Cloud Crowbar 8, it is not possible to put your entire SUSE OpenStack Cloud into “Maintenance Mode” (such as limiting all users to read-only operations on the control plane), as OpenStack does not support this. However when Pacemaker is deployed to manage HA clusters, it should be used to place services and cluster nodes into “Maintenance Mode” before performing maintenance functions on them. For more information, see https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#cha-conf-hawk2.
While the Administration Server is offline, it is not possible to deploy new nodes. However, rebooting the Administration Server has no effect on starting instances or on instances already running.
The consequences a reboot of a Control Node depend on the services running on that node:
Database, Keystone, RabbitMQ, Glance, Nova: No new instances can be started.
Swift: No object storage data is available. If Glance uses Swift, it will not be possible to start new instances.
Cinder, Ceph: No block storage data is available.
Neutron: No new instances can be started. On running instances the network will be unavailable.
Horizon. Horizon will be unavailable. Starting and managing instances can be done with the command line tools.
Whenever a Compute Node is rebooted, all instances running on that particular node will be shut down and must be manually restarted. Therefore it is recommended to “evacuate” the node by migrating instances to another node, before rebooting it.
In case you need to restart your complete SUSE OpenStack Cloud (after a complete shut down or a power outage), ensure that the external Ceph cluster is started, available and healthy. Start then the nodes and services in the following order:
Control Node/Cluster on which the Database is deployed
Control Node/Cluster on which RabbitMQ is deployed
Control Node/Cluster on which Keystone is deployed
For Swift:
Storage Node on which the swift-storage
role is deployed
Storage Node on which the swift-proxy
role is deployed
Any remaining Control Node/Cluster. The following additional rules apply:
The Control Node/Cluster on which the neutron-server
role is deployed needs to be started before starting the node/cluster
on which the neutron-l3
role is deployed.
The Control Node/Cluster on which the nova-controller
role is deployed needs to be started before starting the node/cluster
on which Heat is deployed.
Compute Nodes
If multiple roles are deployed on a single Control Node, the services are automatically started in the correct order on that node. If you have more than one node with multiple roles, make sure they are started as closely as possible to the order listed above.
If you need to shut down SUSE OpenStack Cloud, the nodes and services need to be terminated in reverse order than on start-up:
Compute Nodes
Control Node/Cluster on which Heat is deployed
Control Node/Cluster on which the nova-controller
role is deployed
Control Node/Cluster on which the neutron-l3
role is deployed
All Control Node(s)/Cluster(s) on which neither of the following services is deployed: Database, RabbitMQ, and Keystone.
For Swift:
Storage Node on which the swift-proxy
role is
deployed
Storage Node on which the swift-storage
role is
deployed
Control Node/Cluster on which Keystone is deployed
Control Node/Cluster on which RabbitMQ is deployed
Control Node/Cluster on which the Database is deployed
If required, gracefully shut down an external Ceph cluster
Upgrading from SUSE OpenStack Cloud Crowbar 7 to SUSE OpenStack Cloud Crowbar 8 can be done either via a Web interface or from the command line. A “non-disruptive” update is supported when the requirements listed at Non-Disruptive Upgrade Requirements are met. The non-disruptive upgrade provides a fully-functional SUSE OpenStack Cloud operation during most of the upgrade procedure.
If the requirements for a non-disruptive upgrade are not met, the upgrade procedure will be done in normal mode. When live-migration is set up, instances will be migrated to another node before the respective Compute Node is updated to ensure continuous operation.
Make sure that the STONITH mechanism in your cloud does not rely on the state of the Administration Server (for example, no SBD devices are located there, and IPMI is not using the network connection relying on the Administration Server). Otherwise, this may affect the clusters when the Administration Server is rebooted during the upgrade procedure.
When starting the upgrade process, several checks are performed to determine whether the SUSE OpenStack Cloud is in an upgradeable state and whether a non-disruptive update would be supported:
All nodes need to have the latest SUSE OpenStack Cloud Crowbar 7 updates and the latest SLES 12 SP2 updates installed. If this is not the case, refer to Section 11.4.1, “Deploying Node Updates with the Updater Barclamp” for instructions on how to update.
All allocated nodes need to be turned on and have to be in state “ready”.
All barclamp proposals need to have been successfully deployed. If a proposal is in state “failed”, the upgrade procedure will refuse to start. Fix the issue or—if possible—remove the proposal.
If the Pacemaker barclamp is deployed, all clusters need to be in a healthy state.
The upgrade will not start when Ceph is deployed via Crowbar. Only external Ceph is supported. Documentation for SUSE Enterprise Storage is available at https://documentation.suse.com/ses/5.5.
Upgrade is only possible if the SQL
Engine
in the Database
barclamp is set to
. For further info, see Section 17.3.2, “Preparing PostgreSQL-Based SUSE OpenStack Cloud Crowbar 7 for Upgrade”
The following repositories need to be available on a server that is accessible from the Administration Server. The HA repositories are only needed if you have an HA setup. It is recommended to use the same server that also hosts the respective repositories of the current version.
SUSE-OpenStack-Cloud-Crowbar-8-Pool |
SUSE-OpenStack-Cloud-Crowbar-8-Update |
SLES12-SP3-Pool |
SLES12-SP3-Update |
SLE12-HA12-SP3-Pool (for HA setups only)
|
SLE12-HA12-SP3-Update (for HA setups only)
|
Do not add repositories to the SUSE OpenStack Cloud repository configuration. This needs to be done during the upgrade procedure.
A non-disruptive upgrade is not supported if Cinder has been
deployed with the raw devices
or local
file
back-end. In this case, you have to perform a regular
upgrade, or change the Cinder back-end for a non-disruptive
upgrade.
If SUSE Enterprise Storage is deployed using Crowbar, migrate it to an external cluster. You may want to upgrade SUSE Enterprise Storage, refer to https://documentation.suse.com/ses/5.5/single-html/ses-deployment/#ceph-upgrade-4to5crowbar.
Run the command nova-manage db archive_deleted_rows
to purge deleted instances from the database table. This can significanly reduce time required for the database migration procedure.
Run the commands cinder-manage db purge
and heat-manage purge_deleted
to purge database entries that are marked as deleted.
All Control Nodes need to be set up highly available.
A non-disruptive upgrade is not supported if the Cinder
has been deployed with the raw devices
or
local file
back-end. In this case, you have to perform
a regular upgrade, or change the Cinder back-end for a
non-disruptive upgrade.
A non-disruptive upgrade is prevented if the
cinder-volume
service is placed on Compute Node. For a
non-disruptive upgrade, cinder-volume
should either be
HA-enabled or placed on non-compute nodes.
A non-disruptive upgrade is prevented if manila-share
service is placed on a Compute Node. For more information, see Section 12.15, “Deploying Manila”
Live-migration support needs to be configured and enabled for the Compute Nodes. The amount of free resources (CPU and RAM) on the Compute Nodes needs to be sufficient to evacuate the nodes one by one.
In case of a non-disruptive upgrade, Glance must be configured as a
shared storage if the File
.
For a non-disruptive upgrade, only KVM-based Compute Nodes with
the nova-computer-kvm
role are allowed in SUSE OpenStack Cloud Crowbar 7.
Non-disruptive upgrade is limited to the following cluster configurations:
Single cluster that has all supported controller roles on it
Two clusters where one only has
neutron-network
and the other one has the
rest of the controller roles.
Two clusters where one only has
neutron-server
plus
neutron-network
and the other one has the
rest of the controller roles.
Two clusters, where one cluster runs the database and RabbitMQ
Three clusters, where one cluster runs database and RabbitMQ,
another cluster runs APIs, and the third cluster has the
neutron-network
role.
If your cluster configuration is not supported by the non-disruptive upgrade procedure, you can still perform a normal upgrade.
Upgrading SUSE OpenStack Cloud Crowbar 7 is only possible when it uses MariaDB deployed with the Database barclamp. This means that before you can proceed with upgrading SUSE OpenStack Cloud Crowbar 7, you must migrate PostgreSQL to MariaDB first. The following description covers several possible scenarios.
Install the latest
maintenance updates on SUSE OpenStack Cloud Crowbar 7. In the
Crowbar Web interface, switch to the barclamp. You should
see the new mysql-server
role in the
section. Do not change the
sql_engine
at this point. Add your Database Node or cluster to the
mysql-server
role and apply the
barclamp. MariaDB is now deployed and running, but it is still not used
as a back end for OpenStack services.
Follow Procedure 17.1, “Data Migration” to migrate the data from PostgreSQL to MariaDB.
Run the /opt/dell/bin/prepare-mariadb
script on the Admin Node to prepare the MariaDB instance by creating
the required users, databases, and tables.
After the script is finished, you'll find a list of all databases and
URLs that are ready for data migration in the
/etc/pg2mysql/databases.yaml
located on one of the Database Nodes. The script's output
may look as follows:
Preparing node d52-54-77-77-01-01.vo6.cloud.suse.de Adding recipe[database::pg2mariadb_preparation] to run_list Running chef-client on d52-54-77-77-01-01.vo6.cloud.suse.de... Log: /var/log/crowbar/db-prepare.chef-client.log on d52-54-77-77-01-01.vo6.cloud.suse.de Run time: 444.725193199s Removing recipe[database::pg2mariadb_preparation] from run_list Prepare completed for d52-54-77-77-01-01.vo6.cloud.suse.de Summary of used databases: /etc/pg2mysql/databases.yaml on d52-54-77-77-01-01.vo6.cloud.suse.de
The Summary of used databases:
line shows the
exact location of the /etc/pg2mysql/databases.yaml
file.
The /etc/pg2mysql/databases.yaml
file contains a
list of databases along with their source and target connection strings:
keystone: source: postgresql://keystone:vZn3nfxXzv97@192.168.243.87/keystone target: mysql+pymysql://keystone:vZn3nfxXzv97@192.168.243.88/keystone?charset=utf8 glance: source: postgresql://glance:cOau7NhaA54N@192.168.243.87/glance target: mysql+pymysql://glance:cOau7NhaA54N@192.168.243.88/glance?charset=utf8 cinder: source: postgresql://cinder:idRll2gJPodv@192.168.243.87/cinder target: mysql+pymysql://cinder:idRll2gJPodv@192.168.243.88/cinder?charset=utf8
Install the python-psql2mysql package on the Database Node
(preferably the one with the /etc/pg2mysql/databases.yaml
file).
To determine whether the PostgreSQL databases contain data that cannot
be migrated to MariaDB, run psql2mysql
with the
precheck
option:
tux >
psql2mysql \
--source postgresql://keystone:vZn3nfxXzv97@192.168.243.87/keystone \
--target mysql+pymysql://keystone:vZn3nfxXzv97@192.168.243.88/keystone?charset=utf8 \
precheck
To run precheck operation on all databases in a single operation, use the
--batch
option and the
/etc/pg2mysql/databases.yaml
file as follows:
tux >
psql2mysql --batch /etc/pg2mysql/databases.yaml precheck
If the precheck indicates that there is data that cannot be imported into MariaDB, modify the offending data manually to fix the problems. The example below shows what an output containing issues may look like:
tux >
psql2mysql --source postgresql://cinder:idRll2gJPodv@192.168.243.86/cinder precheck
Table 'volumes' contains 4 Byte UTF8 characters which are incompatible with the 'utf8' encoding used by MariaDB
The following rows are affected:
+-----------------------------------------+-----------------+-------+
| Primary Key | Affected Column | Value |
+-----------------------------------------+-----------------+-------+
| id=5c6b0274-d18d-4153-9fda-ef3d74ab4500 | display_name | 💫 |
+-----------------------------------------+-----------------+-------+
Error during prechecks. 4 Byte UTF8 characters found in the source database.
Stop chef-client services on the nodes, to prevent regular runs of
chef-client from starting database-related OpenStack services again. To
do this from the Admin Node, you can use the knife ssh roles:dns-client systemctl
stop chef-client
command. Stop
all OpenStack services that make use of the database to prevent them from
writing new data during the migration.
If you want to perform a dry run of the migration procedure, you can run the
psql2mysql migrate
without stopping the
database-related OpenStack services. This way, if the test migration
fails due to errors that weren't caught by the precheck procedure, you
can fix them with OpenStack services still running, thus minimizing the
required downtime. When you perform the actual migration, the data in
the target databases will be replaced with the latest one in the source databases.
After the test migration and before the actual migration, it is recommended to run the
psql2mysql purge-tables
command to purge tables in
the target database. While this step is optional, it speeds up the
migration process.
On an HA setup, shut down all services that make use of the
database. To do this, use the crm
command for
example:
crm resource stop apache2 keystone cinder-api glance-api \ neutron-server swift-proxy nova-api magnum-api sahara-api heat-api ceilometer-collector
If the Manage stateless active/active services with
Pacemaker
option in the Pacemaker barclamp is set to
false
, the OpenStack services must be stopped on each
cluster node using the systemctl
command.
From this point, OpenStack services become unavailable.
You can now migrate databases using the psql2mysql tool. However, before performing the migration, make sure that target database nodes have enough free space to accommodate the migrated data. To upgrade a single database, use the following command format:
tux >
psql2mysql \
--source postgresql://neutron:secret@192.168.1.1/neutron \
--target mysql+pymysql://neutron:evenmoresecret@192.168.1.2/neutron?charset=utf8 \
migrate
To migrate all databases in one operation, use the
--batch
option and the
/etc/pg2mysql/databases.yaml
file as follows:
tux >
psql2mysql --batch /etc/pg2mysql/databases.yaml migrate
In the Crowbar Web interface, switch to the sql_engine
to mysql
. Apply the
barclamp. After this step, OpenStack services should be running again and
reconfigured to use the MariaDB database back end.
To prevent the PostgreSQL-related chef code from running, unassign the values
from database-server
role in the
barclamp, and apply the barclamp.
Start chef-client services on the nodes again.
Stop PostgreSQL on the Database Nodes. Uninstall PostgreSQL packages.
To stop the postgresql
service, run the following
command on one cluster node:
tux >
crm resource stop postgresqltux >
crm resource stop fs-postgresqltux >
crm resource stop drbd-postgresql
Run the last command only if the previous setup used DRBD.
Remove the packages on all cluster nodes:
root #
zypper rm postgresql94 postgresql94-server
If you choose not to upgrade to SUSE OpenStack Cloud Crowbar 8 right away, delete unused pacemaker resource from one cluster node:
tux >
crm conf delete drbd-postgresqltux >
crm conf delete fs-postgresqltux >
crm conf delete postgresql
Run the crm conf delete drbd-postgresql
command only
if the cloud setup your are upgrading uses DRBD.
If DRBD is not used as a backend for RabbitMQ, it is possible to remove it at this point, using the following command:
tux >
sudo zypper rm drbd drbd-utils
You can then reclaim the disk space used by Crowbar for DRBD. To do this, edit the node data using knife
:
tux >
knife node edit -a DRBD_NODE
Search for claimed_disks
and remove the entry with owner set to LVM_DRBD
.
Otherwise, skip this step until after the full upgrade is done, since the RabbitMQ setup will be automatically switched from DRBD during the upgrade procedure.
Before your proceed, extend the 2-node cluster with additional node that has no role assigned to it. Make sure that the new node has enough memory to serve as a Control Node.
In Crowbar Web interface, switch to the allow_larger_cluster
option, and set it value to true
. Note that this
is relevant only for DRBD clusters.
Add the pacemaker-cluster-member
role to the new node
and apply the barclamp.
Proceed with the migration procedure as described in Section 17.3.2.1, “Non-HA Setup or HA Setup with More Than 2 Nodes in the Cluster and PostgreSQL Database Backend”.
The Web interface features a wizard that guides you through the upgrade procedure.
You can cancel the upgrade process by clicking
. Note that the upgrade operation can be canceled only before the Administration Server upgrade is started. When the upgrade has been canceled, the nodes return to the ready state. However any user modifications must be undone manually. This includes reverting repository configuration.
To start the upgrade procedure, open the Crowbar Web interface on the Administration Server and choose http://192.168.124.10/upgrade/
.
On the first screen of the Web interface you will run preliminary checks, get information about the upgrade mode and start the upgrade process.
Perform the preliminary checks to determine whether the upgrade
requirements are met by clicking Preliminary Checks
.
The Web interface displays the progress of the checks. Make sure all checks are passed (you should see a green marker next to each check). If errors occur, fix them and run the
again. Do not proceed until all checks are passed.
When all checks in the previous step have passed, Upgrade
Mode
shows the result of the upgrade analysis. It will indicate
whether the upgrade procedure will continue in non-disruptive or in
normal mode.
To start the upgrade process, click
.
While the upgrade of the Administration Server is prepared, the upgrade wizard
prompts you to crowbarctl backup restore
NAME
command.
Check that the repositories required for upgrading the Administration Server are available and updated. To do this, click the Chapter 5, Software Repository Setup of the Deployment Guide. Run the checks again, and click .
button. If the checks fail, add the software repositories as described inClick
to upgrade and reboot the admin node. Note that this operation may take a while. When the Administration Server has been updated, click .Check that the repositories required for upgrading all nodes are available and updated. To do this click the Chapter 5, Software Repository Setup of the Deployment Guide. Run the checks again, and click .
button. If the check fails, add the software repositories as described inStop the OpenStack services. Before you proceed, be aware that no changes can be made to your cloud during and after stopping the services. The OpenStack API will not be available until the upgrade process is completed. When you are ready, click . Wait until the services are stopped and click .
Before upgrading the nodes, the wizard prompts you to
. The MariaDB database backup will be stored on the Administration Server. It can be used to restore the database in case something goes wrong during the upgrade. To back up the database, click . When the backup operation is finished, click .Start the upgrade by clicking
. The number of nodes determines how long the upgrade process will take. When the upgrade is completed, press to return to the Dashboard.With this first maintenance update, only systems already using MariaDB as their OpenStack database will be able to upgrade. In a future maintenance update, there will be a way to migrate from PostgreSQL to MariaDB so PostgreSQL users will be able to upgrade.
If an error occurs during the upgrade process, the wizard displays a message with a description of the error and a possible solution. After fixing the error, re-run the step where the error occurred.
The upgrade procedure on the command line is performed by using the program
crowbarctl
. For general help, run crowbarctl
help
. To get help on a certain subcommand, run
crowbarctl COMMAND help
.
To review the process of the upgrade procedure, you may call
crowbarctl upgrade status
at any time. Steps may have
three states: pending
, running
, and
passed
.
To start the upgrade procedure from the command line, log in to the
Administration Server as root
.
Perform the preliminary checks to determine whether the upgrade requirements are met:
root #
crowbarctl upgrade prechecks
The command's result is shown in a table. Make sure the column
precheck
command
afterwards. Do not proceed before all checks are passed.
root #
crowbarctl upgrade prechecks
+-------------------------------+--------+----------+--------+------+
| Check ID | Passed | Required | Errors | Help |
+-------------------------------+--------+----------+--------+------+
| network_checks | true | true | | |
| cloud_healthy | true | true | | |
| maintenance_updates_installed | true | true | | |
| compute_status | true | false | | |
| ha_configured | true | false | | |
| clusters_healthy | true | true | | |
+-------------------------------+--------+----------+--------+------+
Depending on the outcome of the checks, it is automatically decided whether the upgrade procedure will continue in non-disruptive or in normal mode.
The non-disruptive update will take longer than an upgrade in normal mode, because it performs certain tasks in parallel which are done sequentially during the non-disruptive upgrade. Live-migrating guests to other Compute Nodes during the non-disruptive upgrade takes additional time.
Therefore, if a non-disruptive upgrade is not a requirement for you, you may want to switch to the normal upgrade mode, even if your setup supports the non-disruptive method. To force the normal upgrade mode, run:
root #
crowbarctl upgrade mode normal
To query the current upgrade mode run:
root #
crowbarctl upgrade mode
To switch back to the non-disruptive mode run:
root #
crowbarctl upgrade mode non_disruptive
It is possible to call this command at any time during the upgrade
process until the services
step is started. After
that point the upgrade mode can no longer be changed.
Prepare the nodes by transitioning them into the “upgrade” state and stopping the chef daemon:
root #
crowbarctl upgrade prepare
Depending of the size of your SUSE OpenStack Cloud deployment, this step may take
some time. Use the command crowbarctl upgrade status
to monitor the status of the process named
steps.prepare.status
. It needs to be in state
passed
before you proceed:
root #
crowbarctl upgrade status
+--------------------------------+----------------+
| Status | Value |
+--------------------------------+----------------+
| current_step | backup_crowbar |
| current_substep | |
| current_substep_status | |
| current_nodes | |
| current_node_action | |
| remaining_nodes | |
| upgraded_nodes | |
| crowbar_backup | |
| openstack_backup | |
| suggested_upgrade_mode | non_disruptive |
| selected_upgrade_mode | |
| compute_nodes_postponed | false |
| steps.prechecks.status | passed |
| steps.prepare.status | passed |
| steps.backup_crowbar.status | pending |
| steps.repocheck_crowbar.status | pending |
| steps.admin.status | pending |
| steps.repocheck_nodes.status | pending |
| steps.services.status | pending |
| steps.backup_openstack.status | pending |
| steps.nodes.status | pending |
+--------------------------------+----------------+
Create a backup of the existing Administration Server installation. In case something
goes wrong during the upgrade procedure of the Administration Server you can restore
the original state from this backup with the command crowbarctl
backup restore NAME
root #
crowbarctl upgrade backup crowbar
To list all existing backups including the one you have just created, run the following command:
root #
crowbarctl backup list
+----------------------------+--------------------------+--------+---------+
| Name | Created | Size | Version |
+----------------------------+--------------------------+--------+---------+
| crowbar_upgrade_1534864741 | 2018-08-21T15:19:03.138Z | 219 KB | 4.0 |
+----------------------------+--------------------------+--------+---------+
This step prepares the upgrade of the Administration Server by checking the availability of the update and pool repositories for SUSE OpenStack Cloud Crowbar 8 and SUSE Linux Enterprise Server 12 SP3. Run the following command:
root #
crowbarctl upgrade repocheck crowbar
+----------------------------------------+-------------------------------------+-----------+
| Repository | Status | Type |
+----------------------------------------+-------------------------------------+-----------+
| SLE12-SP3-HA-Pool | missing (x86_64), inactive (x86_64) | ha |
| SLE12-SP3-HA-Updates | available | ha |
| SLES12-SP3-Pool | available | os |
| SLES12-SP3-Updates | available | os |
| SUSE-OpenStack-Cloud-Crowbar-8-Pool | available | openstack |
| SUSE-OpenStack-Cloud-Crowbar-8-Updates | available | openstack |
+----------------------------------------+-------------------------------------+-----------+
The output above indicates that the SLE12-SP3-HA-Pool
repository is missing, because it has
not yet been added to the Crowbar configuration. To add it to the
Administration Server proceed as follows.
Note that this step is for setting up the repositories for the Administration Server, not for the nodes in SUSE OpenStack Cloud (this will be done in a subsequent step).
Start yast repositories
and proceed with
. Replace the repositories
SLES12-SP2-Pool
and
SLES12-SP2-Updates
with the respective SP3
repositories.
If you prefer to use zypper over YaST, you may alternatively make the
change using zypper mr
.
Next, replace the SUSE-OpenStack-Cloud-7
update and
pool repositories with the respective SUSE OpenStack Cloud Crowbar 8
versions.
Check for other (custom) repositories. All SLES SP2 repositories need to be replaced with the respective SLES SP3 version. In case no SP3 version exists, disable the repository—the respective packages from that repository will be deleted during the upgrade.
Once the repository configuration on the Administration Server has been updated, run the command to check the repositories again. If the configuration is correct, the result should look like the following:
root #
crowbarctl upgrade repocheck crowbar
+---------------------+----------------------------------------+
| Status | Value |
+---------------------+----------------------------------------+
| os.available | true |
| os.repos | SLES12-SP3-Pool |
| | SLES12-SP3-Updates |
| openstack.available | true |
| openstack.repos | SUSE-OpenStack-Cloud-Crowbar-8-Pool |
| | SUSE-OpenStack-Cloud-Crowbar-8-Updates |
+---------------------+----------------------------------------+
Now that the repositories are available, the Administration Server itself will be
upgraded. The update will run in the background using zypper
dup
. Once all packages have been upgraded, the Administration Server will
be rebooted and you will be logged out. To start the upgrade run:
root #
crowbarctl upgrade admin
After the Administration Server has been successfully updated, the Control Nodes and Compute Nodes will be upgraded. At first the availability of the repositories used to provide packages for the SUSE OpenStack Cloud nodes is tested.
When adding new repositories to the nodes, make sure that the new PTF
repository also contains correct metadata (even if it is empty). To do
this, run the createrepo-cloud-ptf
command.
Note that the configuration for these repositories differs from the one for the Administration Server that was already done in a previous step. In this step the repository locations are made available to Crowbar rather than to libzypp on the Administration Server. To check the repository configuration run the following command:
root #
crowbarctl upgrade repocheck nodes
+---------------------------------+----------------------------------------+
| Status | Value |
+---------------------------------+----------------------------------------+
| ha.available | false |
| ha.repos | SLES12-SP3-HA-Pool |
| | SLES12-SP3-HA-Updates |
| ha.errors.x86_64.missing | SLES12-SP3-HA-Pool |
| | SLES12-SP3-HA- Updates |
| os.available | false |
| os.repos | SLES12-SP3-Pool |
| | SLES12-SP3-Updates |
| os.errors.x86_64.missing | SLES12-SP3-Pool |
| | SLES12-SP3-Updates |
| openstack.available | false |
| openstack.repos | SUSE-OpenStack-Cloud-Crowbar-8-Pool |
| | SUSE-OpenStack-Cloud-Crowbar-8-Updates |
| openstack.errors.x86_64.missing | SUSE-OpenStack-Cloud-Crowbar-8-Pool |
| | SUSE-OpenStack-Cloud-Crowbar-8-Updates |
+---------------------------------+----------------------------------------+
To update the locations for the listed repositories, start yast
crowbar
and proceed as described in Section 7.4, “.
”
Once the repository configuration for Crowbar has been updated, run the command to check the repositories again to determine, whether the current configuration is correct.
root #
crowbarctl upgrade repocheck nodes
+---------------------+----------------------------------------+
| Status | Value |
+---------------------+----------------------------------------+
| ha.available | true |
| ha.repos | SLE12-SP3-HA-Pool |
| | SLE12-SP3-HA-Updates |
| os.available | true |
| os.repos | SLES12-SP3-Pool |
| | SLES12-SP3-Updates |
| openstack.available | true |
| openstack.repos | SUSE-OpenStack-Cloud-Crowbar-8-Pool |
| | SUSE-OpenStack-Cloud-Crowbar-8-Updates |
+---------------------+----------------------------------------+
If the upgrade is done in normal mode (prechecks compute_status and ha_configured have not been passed), you need to shut down all running instances now.
To PXE boot new nodes, an additional SUSE Linux Enterprise Server 12 SP3 repository—a copy of the installation system— is required. Although not required during the upgrade procedure, it is recommended to set up this directory now. Refer to Section 5.1, “Copying the Product Media Repositories” for details. If you had also copied the SUSE OpenStack Cloud Crowbar 6 installation media (optional), you may also want to provide the SUSE OpenStack Cloud Crowbar 8 the same way.
Once the upgrade procedure has been successfully finished, you may
delete the previous copies of the installation media in
/srv/tftpboot/suse-12.2/x86_64/install
and
/srv/tftpboot/suse-12.2/x86_64/repos/Cloud
.
To ensure the status of the nodes does not change during the upgrade process, the majority of the OpenStack services will be stopped on the nodes. As a result, the OpenStack API will no longer be accessible. The instances, however, will continue to run and will also be accessible. Run the following command:
root #
crowbarctl upgrade services
This step takes a while to finish. Monitor the process by running
crowbarctl upgrade status
. Do not proceed before
steps.services.status
is set to
passed
.
The last step before upgrading the nodes is to make a backup of the OpenStack PostgreSQL database. The database dump will be stored on the Administration Server and can be used to restore the database in case something goes wrong during the upgrade.
root #
crowbarctl upgrade backup openstack
The final step of the upgrade procedure is upgrading the nodes. To start the process, enter:
root #
crowbarctl upgrade nodes all
The upgrade process runs in the background and can be queried with
crowbarctl upgrade status
. Depending on the size of
your SUSE OpenStack Cloud it may take several hours, especially when performing a
non-disruptive update. In that case, the Compute Nodes are updated
one-by-one after instances have been live-migrated to other nodes.
Instead of upgrading all nodes you may also upgrade
the Control Nodes first and individual Compute Nodes afterwards. Refer to
crowbarctl upgrade nodes --help
for details. If you
choose this approach, you can use the crowbarctl upgrade
status
command to monitor the upgrade process. The output of
this command contains the following entries:
The current action applied to the node.
Shows the current substep of the node upgrade step. For example,
for the crowbarctl upgrade nodes controllers
,
the current_substep
entry displays the
controller_nodes
status when upgrading controllers.
After the controllers have been upgraded, the
steps.nodes.status
entry in the output displays the
running
status. Check then the status of the
current_substep_status
entry. If it displays
finished
, you can move to the next step of upgrading
the Compute Nodes.
Postponing the Upgrade
It is possible to stop the upgrade of compute nodes and postpone their upgrade with the command:
root #
crowbarctl upgrade nodes postpone
After the upgrade of compute nodes is postponed, you can go to Crowbar Web interface, check the configuration. You can also apply some changes, provided they do not affect the Compute Nodes. During the postponed upgrade, all OpenStack services should be up and running. Compute Nodes are still running old version of services.
To resume the upgrade, issue the command:
root #
crowbarctl upgrade nodes resume
And finish the upgrade with either crowbarctl upgrade nodes
all
or upgrade nodes one node by one with crowbarctl
upgrade nodes NODE_NAME
.
When upgrading individual Compute Nodes using the crowbarctl
upgrade nodes NODE_NAME
command, the
current_substep_status
entry changes to
node_finished
when the upgrade of a single node is
done. After all nodes have been upgraded, the
current_substep_status
entry displays finished
.
If an error occurs during the upgrade process, the output of the
crowbarctl upgrade status
provides a detailed
description of the failure. In most cases, both the output and the error
message offer enough information for fixing the issue. When the problem has
been solved, run the previously-issued upgrade command to resume the
upgrade process.
It is possible to select more Compute Nodes for selective upgrade instead of just one. Upgrading multiple nodes simultaneously significantly reduces the time required for the upgrade.
To upgrade multiple nodes simultaneously, use the following command:
root #
crowbarctl upgrade nodes NODE_NAME_1,NODE_NAME_2,NODE_NAME_3
Node names can be separated by comma, semicolon, or space. When using space as separator, put the part containing node names in quotes.
Use the following command to find the names of the nodes that haven't been upgraded:
root #
crowbarctl upgrade status nodes
Since the simultaneous upgrade is intended to be non-disruptive, all Compute Nodes targeted for a simultaneous upgrade must be cleared of any running instances.
You can check what instances are running on a specific node using the following command:
tux >
nova list --all-tenants --host NODE_NAME
This means that it is not possible to pick an arbitrary number of
Compute Nodes for the simultaneous upgrade operation: you have to make sure
that it is possible to live-migrate every instance away from the batch of
nodes that are supposed to be upgraded in parallel. In case of high load
on all Compute Nodes, it might not be possible to upgrade more than one node
at a time. Therefore, it is recommended to perform the following steps for
each node targeted for the simultaneous upgrade prior to running the
crowbarctl upgrade nodes
command.
Disable the Compute Node so it's not used as a target during live-evacuation of any other node:
tux >
openstack compute service set --disable "NODE_NAME" nova-compute
Evacuate all running instances from the node:
tux >
nova host-evacuate-live "NODE_NAME"
After completing these steps, you can perform a simultaneous upgrade of the selected nodes.
Check for empty, broken, and not signed repositories in the Administration Server
upgrade log file /var/log/crowbar/admin-server-upgrade.log
. Fix the
repository setup. Upgrade then remaining packages manually to SUSE Linux Enterprise Server 12 SP3
and SUSE OpenStack Cloud 8 using the command zypper dup
. Reboot the Administration Server.
Timeouts for most upgrade operations can be adjusted in the
/etc/crowbar/upgrade_timeouts.yml
file. If the
file doesn't exist, use the following template, and modify it to your needs:
:prepare_repositories: 120 :pre_upgrade: 300 :upgrade_os: 1500 :post_upgrade: 600 :shutdown_services: 600 :shutdown_remaining_services: 600 :evacuate_host: 300 :chef_upgraded: 1200 :router_migration: 600 :lbaas_evacuation: 600 :set_network_agents_state: 300 :delete_pacemaker_resources: 600 :delete_cinder_services: 300 :delete_nova_services: 300 :wait_until_compute_started: 60 :reload_nova_services: 120 :online_migrations: 1800
The following entries may require higher values (all values are specified in seconds):
upgrade_os
Time allowed for upgrading all packages of one node.
chef_upgraded
Time allowed for initial
crowbar_join
and chef-client
run on a node that has been upgraded and rebooted.
evacuate_host
Time allowed for live migrate all VMs from a host.
The problem may occur when it is not possible to live migrate certain
VMs anywhere. It may be necessary to shut down or suspend other VMs to
make room for migration. Note that the Bash shell script that starts
the live migration for the Compute Node is executed from the
Control Node. An error message generated by the crowbarctl
upgrade status
command contains the exact names of both
nodes. Check the /var/log/crowbar/node-upgrade.log
file on the Control Node for the information that can help you with
troubleshooting. You might also need to check OpenStack logs in
/var/log/nova
on the Compute Node as well as on the
Control Nodes.
It is possible that live-migration of a certain VM takes too long. This
can happen if instances are very large or network connection between
compute hosts is slow or overloaded. If this case, try to raise the
global timeout in
/etc/crowbar/upgrade_timeouts.yml
.
We recommend to perform the live migration manually first. After it is
completed successfully, call the crowbarctl upgrade
command again.
The following commands can be helpful for analyzing issues with live migrations:
nova server-migration-list nova server-migration-show nova instance-action-list nova instance-action
Note that these commands require OpenStack administrator privileges.
The following log files may contain useful information:
/var/log/nova/nova-compute
on the Compute Nodes
that the migration is performed from and to.
/var/log/nova/*.log
(especially log files for the
conductor, scheduler and placement services) on the Control Nodes.
It can happen that active instances and instances with heavy
loads cannot be live migrated in a reasonable time. In that case, you
can abort a running live-migration operation using the nova
live-migration-abort MIGRATION-ID
command. You can then perform the upgrade of the specific node at a
later time.
Alternatively, it is possible to force the completion of
the live migration by using the nova
live-migration-force-complete
MIGRATION-ID
command. However,
this might pause the instances for a prolonged period of time and have
a negative impact on the workload running inside the instance.
Possible reasons include an incorrect repository setup or package
conflicts. Check the /var/log/crowbar/node-upgrade.log
log file on the
affected node. Check the repositories on node using the zypper
lr
command. Make sure the required repositories are
available. To test the setup, install a package manually or run the
zypper dup
command (this command is executed by the
upgrade script). Fix the repository setup and run the failed upgrade
step again. If custom package versions or version locks are in place,
make sure that they don't interfere with the zypper dup
command.
In some cases, a node can take too long to reboot causing a timeout. We recommend to check the node manually, make sure it is online, and repeat the step.
crowbarctl upgrade nodes node_1,node_2,...,node_N
,
but less then N were actually upgraded.
If the live migration cannot be performed for certain nodes due to a timeout, Crowbar upgrades only the nodes that it was able to live-evacuate in the specified time. Because some nodes have been upgraded, it is possible that more resources will be available for live-migration when you try to run this step again. See also Node upgrade has failed during live migration. .
An unsupported entry in the configuration file may prevent a service
from starting. This causes the node to fail at the initial
chef client run stage. Checking the
/var/log/crowbar/crowbar_join/chef.*
log files on
the node is a good starting point.
Crowbar Web interface is accessible only when an upgrade is completed or
when it is postponed. Postponing the upgrade can be done only after
upgrading all Control Nodes using the crowbarctl upgrade nodes
postpone
command. You can then access Crowbar and
save your modifications. Before you can continue with the upgrade of
rest of the nodes, resume the upgrade using the crowbarctl
upgrade nodes resume
command.
Check the /var/log/crowbar/node-upgrade.log
file on
the node that performs the router evacuation (it should be mentioned in
the error message). The ID of the router that failed to migrate (or the
affected network port) is logged to
/var/log/crowbar/node-upgrade.log
. Use the
OpenStack CLI tools to check the state of the affected router and
its ports. Fix manually, if necessary. This can be done by bringing the
router or port up and down again. The following
commands can be useful for solving the issue:
openstack router show ID openstack port list --router ROUTER-ID openstack port show PORT-ID openstack port set
Resume the upgrade by running the failed upgrade step again to continue with the router migration.
crowbarctl upgrade nodes
controllers
.
In the current upgrade implementation, OpenStack nodes are divided
into Compute Nodes and other nodes. The crowbarctl upgrade nodes
controllers
command starts the upgrade of all the nodes that
do not host compute services. This includes the controllers.
The following procedure assumes that there is at least one Compute Node already running. Otherwise, see Section 17.5, “Bootstrapping Compute Plane”.
If the Compute Node failed, it should have been fenced. Verify that this is
the case. Otherwise, check /var/log/pacemaker.log
on
the Designated Coordinator to determine why the Compute Node was not fenced.
The most likely reason is a problem with STONITH devices.
Determine the cause of the Compute Node's failure.
Rectify the root cause.
Boot the Compute Node again.
Check whether the crowbar_join
script ran
successfully on the Compute Node. If this is not the case, check the log
files to find out the reason. Refer to
Section 19.2, “On All Other Crowbar Nodes” to find the exact
location of the log file.
If the chef-client
agent triggered by
crowbar_join
succeeded, confirm that the
pacemaker_remote
service is up and running.
Check whether the remote node is registered and considered healthy by the
core cluster. If this is not the case check
/var/log/pacemaker.log
on the Designated Coordinator
to determine the cause. There should be a remote primitive running on the
core cluster (active/passive). This primitive is responsible for
establishing a TCP connection to the
pacemaker_remote
service on port 3121 of the
Compute Node. Ensure that nothing is preventing this particular TCP
connection from being established (for example, problems with NICs,
switches, firewalls etc.). One way to do this is to run the following
commands:
tux >
lsof -i tcp:3121tux >
tcpdump tcp port 3121
If Pacemaker can communicate with the remote node, it should start the
nova-compute
service on it as part of the cloned
group cl-g-nova-compute
using the NovaCompute OCF
resource agent. This cloned group will block startup of
nova-evacuate
until at least one clone is
started.
A necessary, related but different procedure is described in Section 17.5, “Bootstrapping Compute Plane”.
It may happen that NovaCompute
has been launched
correctly on the Compute Node by lrmd
, but the
openstack-nova-compute
service is still not
running. This usually happens when nova-evacuate
did not run correctly.
If nova-evacuate
is not
running on one of the core cluster nodes, make sure that the service is
marked as started (target-role="Started"
). If this is
the case, then your cloud does not have any Compute Nodes already running as
assumed by this procedure.
If nova-evacuate
is started but it is
failing, check the Pacemaker logs to determine the cause.
If nova-evacuate
is started and
functioning correctly, it should call Nova's
evacuate
API to release resources used by the
Compute Node and resurrect elsewhere any VMs that died when it failed.
If openstack-nova-compute
is running, but VMs are
not booted on the node, check that the service is not disabled or
forced down using the nova service-list
command. In
case the service is disabled, run the nova service-enable
SERVICE_ID
command. If the service is
forced down, run the following commands:
tux >
fence_nova_param () { key="$1" cibadmin -Q -A "//primitive[@id="fence-nova"]//nvpair[@name='$key']" | \ sed -n '/.*value="/{s///;s/".*//;p}' }tux >
fence_compute \ --auth-url=`fence_nova_param auth-url` \ --endpoint-type=`fence_nova_param endpoint-type` \ --tenant-name=`fence_nova_param tenant-name` \ --domain=`fence_nova_param domain` \ --username=`fence_nova_param login` \ --password=`fence_nova_param passwd` \ -n COMPUTE_HOSTNAME \ --action=on
The above steps should be performed automatically after the node is booted. If that does not happen, try the following debugging techniques.
Check the evacuate
attribute for the Compute Node in the
Pacemaker cluster's attrd
service using the
command:
tux >
attrd_updater -p -n evacuate -N NODE
Possible results are the following:
The attribute is not set. Refer to Step 1 in Procedure 17.2, “Procedure for Recovering from Compute Node Failure”.
The attribute is set to yes
. This means that the
Compute Node was fenced, but nova-evacuate
never
initiated the recovery procedure by calling Nova's evacuate API.
The attribute contains a time stamp, in which case the recovery procedure was initiated at the time indicated by the time stamp, but has not completed yet.
If the attribute is set to no
, the recovery procedure
recovered successfully and the cloud is ready for the Compute Node to
rejoin.
If the attribute is stuck with the wrong value, it can be set to
no
using the command:
tux >
attrd_updater -n evacuate -U no -N NODE
After standard fencing has been performed, fence agent
fence_compute
should activate the secondary
fencing device (fence-nova
). It does this by setting
the attribute to yes
to mark the node as needing
recovery. The agent also calls Nova's
force_down
API to notify it that the host is down.
You should be able to see this in
/var/log/nova/fence_compute.log
on the node in the core
cluster that was running the fence-nova
agent at
the time of fencing. During the recovery, fence_compute
tells Nova that the host is up and running again.
If the whole compute plane is down, it is not always obvious how to boot it
up, because it can be subject to deadlock if evacuate attributes are set on
every Compute Node. In this case, manual intervention is
required. Specifically, the operator must manually choose one or more
Compute Nodes to bootstrap the compute plane, and then run the
attrd_updater -n evacuate -U no -N NODE
command for each
of those Compute Nodes to indicate that they do not require the resurrection
process and can have their nova-compute
start up straight
away. Once these Compute Nodes are up, this breaks the deadlock allowing
nova-evacuate
to start. This way, any other nodes that
require resurrection can be processed automatically. If no resurrection is
desired anywhere in the cloud, then the attributes should be set to
no
for all nodes.
If Compute Nodes are started too long after the
remote-*
resources are started on the control plane,
they are liable to fencing. This should be avoided.
Updating MariaDB with Galera must be done manually. Crowbar does not install updates automatically. Updates can be done with Pacemaker or with the CLI. In particular, manual updating applies to upgrades to MariaDB 10.2.17 or higher from MariaDB 10.2.16 or earlier. See MariaDB 10.2.22 Release Notes - Notable Changes.
In order to run the following update steps, the database cluster needs to be up and healthy.
Using the Pacemaker GUI, update MariaDB with the following procedure:
Put the cluster into maintenance mode. Detailed information about the Pacemaker GUI and its operation is available in the https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#cha-conf-hawk2.
Perform a rolling upgrade to MariaDB following the instructions at Upgrading Between Minor Versions with Galera Cluster.
The process involves the following steps:
Stop MariaDB
Uninstall the old versions of MariaDB and the Galera wsrep provider
Install the new versions MariaDB and the Galera wsrep provider
Change configuration options if necessary
Start MariaDB
Run mysql_upgrade
with the
--skip-write-binlog
option
Each node must upgraded individually so that the cluster is always operational.
Using the Pacemaker GUI, take the cluster out of maintenance mode.
When updating with the CLI, the database cluster must be up and healthy. Update MariaDB with the following procedure:
Mark Galera as unmanaged:
crm resource unmanage galera
Or put the whole cluster into maintenance mode:
crm configure property maintenance-mode=true
Pick a node other than the one currently targeted by the loadbalancer and stop MariaDB on that node:
crm_resource --wait --force-demote -r galera -V
Perform updates with the following steps:
Uninstall the old versions of MariaDB and the Galera wsrep provider.
Install the new versions of MariaDB and the Galera wsrep provider. Select the appropriate instructions at Installing MariaDB with zypper.
Change configuration options if necessary.
Start MariaDB on the node.
crm_resource --wait --force-promote -r galera -V
Run mysql_upgrade
with the
--skip-write-binlog
option.
On the other nodes, repeat the process detailed above: stop MariaDB,
perform updates, start MariaDB, run mysql_upgrade
.
Mark Galera as managed:
crm resource manage galera
Or take the cluster out of maintenance mode.
Heat-manage helps manage Heat specific database operations. The associated
database should be periodically purged to save space. The following should
be setup as a cron job on the servers where the heat service is running at
/etc/cron.weekly/local-cleanup-heat
with the following content:
#!/bin/bash su heat -s /bin/bash -c "/usr/bin/heat-manage purge_deleted -g days 14" || :
nova-manage db archive_deleted_rows command will move deleted rows
from production tables to shadow tables. Including
--until-complete
will make the command run continuously
until all deleted rows are archived. It is recommended to setup this task
as /etc/cron.weekly/local-cleanup-nova
on the servers where the nova service is running, with the
following content:
#!/bin/bash su nova -s /bin/bash -c "/usr/bin/nova-manage db archive_deleted_rows --until-complete" || :
Fernet tokens should be rotated frequently for security purposes.
It is recommended to setup this task as a cron job in
/etc/cron.weekly/openstack-keystone-fernet
on the keystone server designated as a master node in a highly
available setup with the following content:
#!/bin/bash su keystone -s /bin/bash -c "keystone-manage fernet_rotate" /usr/bin/keystone-fernet-keys-push.sh 192.168.81.168; /usr/bin/keystone-fernet-keys-push.sh 192.168.81.169;
The IP addresses in the above example, i.e. 192.168.81.168 and 192.168.81.169 are the IP addresses of the other two nodes of a three-node cluster. Be sure to use the correct IP addresses when configuring the cron job. Note that if the master node is offline and a new master is elected, the cron job will need to be removed from the previous master node and then re-created on the new master node. Do not run the fernet_rotate cron job on multiple nodes.
For a non-HA setup, the cron job should be configured at
/etc/cron.weekly/openstack-keystone-fernet
on the keystone server as follows:
#!/bin/bash su keystone -s /bin/bash -c "keystone-manage fernet_rotate"
The purpose of this document is to help set up SSL Support for several services in SUSE OpenStack Cloud. The scope of this document covers all public endpoints in your OpenStack cluster. In most cases you want to have a Secure CA or External CA where your certificates are signed. You will sign with either a public CA or self signed CA, and include x509 extensions for Subject Alt Names since there might be a highly available control plane with alternate names.
This section demonstrates how to create the certificate on the crowbar or admin node of the SUSE OpenStack Cloud Cluster.
To avoid external access to your CA Root Pair, put it on an air-gapped system that is permanently isolated from the internet and unplug any cables from the ethernet port.
Create a directory for your CA Root pair:
# ssh root@crowbar # mkdir -p ~/ssl/root/ca
Create a directory structure and add index.txt
and serial files to act as flat database of all signed certificates:
# cd ~/ssl/root/ca # mkdir certs crl newcerts private csr # chmod 700 private # touch index.txt # echo 1000 > serial
This procedure takes you through the full set up. Note that
when you setup the crowbar server, there is a structure already setup
under /etc/ssl
. This is where SUSE Linux typically
contains the CA cert bundle created through YaST when the SMT server
is set up. However, if you are using an external SMT server
you will not have this.
Copy /etc/ssl/openssl.cnf
file to your setup.
We can use this since it is completely annotated.
# cp /etc/ssl/openssl.cnf ./
Edit the file and change the location variable:
dir = /root/ssl/root/ca # Where everything is kept
Make sure dir
is the directory where we created
/root/ssl/root/ca
.
Create the root key encrypted with AES 256-bit encryption and a password, using 4096 bits for the creation.
# cd ~/ssl/root/ca # openssl genrsa -aes256 -out private/cakey.pem 4096
You will be asked to enter a password here and then verify it.
# chmod 400 private/cakey.pem
Use the root key (cakey.pem
) to create the
root certificate (cacert.pem). Its important to give it a long
expiration since all the certificates signed from it will
become invalid when it expires.
# cd ~/ssl/root/ca # openssl req -config openssl.cnf -key private/cakey.pem -new -x509 -days 10950 -sha256 -extensions v3_ca -out cacert.pem Enter pass phrase for cakey.pem: enteryourpassword You are about to be asked to enter information that will be incorporated into your certificate request. ----- Country Name (2 letter code) [AU]:US State or Province Name []:Idaho Locality Name []:Meridian Organization Name []:SUSEDojo Organizational Unit Name []:dojo Common Name []:susedojo.com Email Address []:admin@susedojo.com # chmod 444 cacert.pem
Verify the certificate has the correct dates of validity and the algorithm used, Issuer, Subject, and x509v3 extensions. The issuer and subject are the same since it is self signed.
# cd ~/ssl/root/ca # openssl x509 -noout -text -in cacert.pem
This section is if you are the perspective certificate authority (CA).
Modify the penssl.cnf
config file and add a
line to the [ v3_req ]
section:
# cd ~/ssl/root/ca # vi openssl.cnf find v3_req Add the following line: subjectAltName = DNS:public.your_server_name.your_domain.com, DNS: cluster-control.your_domain.com At the bottom of the file create section server_cert with the follwing: [ server_cert ] subjectAltName = subjectAltName = DNS:public.your_server_name.your_domain.com, DNS: cluster-control.your_domain.com
The first DNS name would be used if you only have a single node
controller as you need the public URL for that server in your cluster.
For example, public.db8-ae-ed-77-14-9e.susedojo.com
.
If you have a haproxy setup for your cluster or pacemaker, you have a
cluster URL. For example, you may have
public.cluster.your_domain.com
and you need to
have cluster.your_domain.com
and public.cluster.your_domain.com
as Alternative DNS names. This public URL can be used for all
endpoints unless you have multiple High Availability Clusters for
your control plane.
Save and close the file after you have those entered correctly.
Create a key minus the -aes256
option so that
you are not presented with a password each time you restart a
service. (i.e. Apache service) also in 2048 bit so it is quicker to decrypt.
# cd ~/ssl/root/ca # openssl genrsa -out private/susedojo-com.key.pem 2048 # chmod 400 private/susedojo-com.key.pem
Use the private key we just created to create a certificate signing request (CSR). The common name must be a fully qualified domain name (i.e. www.susedojo.com) The Organization Name must be the same as the Organization Name in the CA.
# cd ~/ssl/root/ca # openssl req -config openssl.cnf -key private/susedojo-com.key.pem -new -sha256 -out csr/susedojo-com.csr.pem You are about to be asked to enter information that will be incorporated into your certificate request. ----- Country Name (2 letter code) [XX]:US State or Province Name []:Idaho Locality Name []:Meridian Organization Name []:SUSEDojo Organizational Unit Name []:dojo Common Name []:susedojo.com Email Address []:admin@susedojo.com
You may be prompted for a challenge password and company name. This can be left blank.
Create the certificate using the CA to sign the CSR, using the
server_cert
extension as this will be used
on a server. We will give it one year of validity.
# cd ~/ssl/root/ca # openssl ca -config openssl.cnf -extensions server_cert -days 365 -notext -md sha256 -in csr/susedojo-com.csr.pem -out certs/susedojo-com.cert.pem Using configuration from openssl.cnf Enter pass phrase for /root/ssl/root/ca/private/cakey.pem: Check that the request matches the signature Signature ok Serial Number: 4096 (0x1000) Validity Not Before: Aug 8 04:21:08 2018 GMT Not After: Aug 8 04:21:08 2019 GMT Subject: countryName = US stateOrProvinceName = Idaho organizationName = SUSEDojo organizationalUnitName = dojo commonName = susedojo.com emailAddress = admin@susedojo.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE X509v3 Key Usage: Digital Signature, Non Repudiation, Key Encipherment X509v3 Subject Alternative Name: DNS:public.db8-ae-ed-77-14-9e.susedojo.com Certificate is to be certified until Aug 8 04:21:08 2019 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated # chmod 444 certs/susedojo-com.cert.pem
The index.txt
file should now contain a line
referring to the new certificate that has been created.
For example, the output should look like the following:
V 190808042108Z 1000 unknown /C=US/ST=Idaho/O=SUSEDojo/OU=dojo/CN=susedojo.com/emailAddress=admin@susedojo.com
Enter the following in your terminal:
# openssl x509 -noout -text -in certs/susedojo-com.cert.pem
You will notice the Issuer is the CA and you can also see the Subject Alternative Name as well in the extensions section.
# openssl verify -CAfile cacert.pem certs/susedojo-com.cert.pem certs/susedojo-com.cert.pem: OK
Now you are ready to copy the newly created certificate and key to the control node or controllers in the cluster.
# scp newcerts/1000.pem control:/root/ # scp private/susedojo-com.key control:/root/
Copy them into the right location on the controller host:
# cp susedojo-com.key.pem /etc/keystone/ssl/private # cp 1000.pem /etc/keystone/ssl/certs # cd /etc/keystone/ssl/certs # mv signing_cert.pem signing_cert.pem.todays_date # cp 1000.pem signing_cert.pem # cd /etc/keystone/ssl/private # old signing_key.pem # cp susedojo-com.key.pem signing_key.pem
Rerun the Barclamp for keystone in order to apply this change to the cluster.
Let’s Encrypt
is a free, automated, and open Certificate Authority.
Its Root is trusted by all major operating systems now. For SUSE Linux
Enterprise Server 12 SP3 and higher, the ISRG Root X1 is available in
/etc/ssl/certs/ISRG_Root_X1.pem
. If not, apply the
latest updates for your operating system.
Let’s Encrypt
has several clients to choose from depending on your needs.
For this example, we will be using the acme.sh
client,
which is written in bash and gives us greater flexibility and ease in our
solution.
The next steps walk you through the installation of acme.sh
and the issue
of a certificate with Let’s Encrypt
followed by the automated load
of the certificate in OpenStack for the various API endpoints available.
Login to your crowbar/admin server change to the root directory.
Create a new directory for letsencrypt and clone the
acme.sh
repo:
# mkdir letsencrypt # cd letsencrypt # git clone https://github.com/Neilpang/acme.sh.git # cd acme.sh
The system is prepared for installing acme.sh
.
Install socat
:
# export BRANCH=2 #this makes sure you are using the v2 api version of letsencrypt # zypper in -y socat
Install acme.sh
:
# ./acme.sh --install
After the install of acme.sh
is finished, you
should see a new directory /root/.acme.sh/
where
acme.sh
lives and all of its environment,
account info, and certificates are stored.
We recommend using this as a backup location if you are using a
backup tool.
OpenStack and wildcard SSL uses the DNS validation method by validating your domain using a TXT record that can either be added manually or using the many (over 3 dozen) available DNS API’s.
It is important to a wildcard certificate as you have the ability to use the same one for all of your public API endpoints in the OpenStack Cloud environment. Additional Cloud Native services like Kubernetes can also take advantage of it.
The manual DNS mode is a method that displays the DNS records that have to be created in your DNS servers. It is beneficial to automate the injection of DNS records as the maximum days a certificate is viable is 60 days. In order to issue your wildcard certificate, the command without optional settings is:
# acme.sh --issue -d yourdomain.com -d *.yourdomain.com --dns
To debug or test, add the following optional settings:
# acme.sh --debug --test –issue -d yourdomain.com -d *.yourdomain.com --dns
A message returns. For example:
Add the following TXT record: Domain: '_acme-challenge.yourdomain.com' TXT value: 'KZvgq3MpOCjUNW7Uzz5nE5kkFdplNk66WGfxE9-H63k' Please be aware that you prepend_acme-challenge.
before your domain so the resulting subdomain will be:_acme-challenge.yourdomain.com
Using this information, you are ready to insert this TXT record into your DNS.
When setting this up for SUSE OpenStack Cloud with Crowbar, you need
to have your external DNS server appended to /etc/resolv.conf
in order to resolve as crowbar has its own internal
DNS. It is not enough to change it in the barclamp as you need the
DNS server entry to be at the top of the list in resolv.conf
.
Crowbar returns to the default after a period of time.
Keep in mind that if you want to automate this step every 90 days
then you need to ensure the resolv.conf
changes
every time to bypass the local crowbar DNS Server.
In order to set up TXT record in bind DNS, edit the zone file so it looks like the following example:
yourdomain.com. IN NS admin.yourdomain.com. _acme-challenge.yourdomain.com. IN TXT "xxxx...your TXT value string here"
Restart your named services for bind
.
Issue the acme-challenge verification of the previous step with the following command:
# acme.sh --renew -d yourdomain.com
If the DNS validation is okay, acme.sh
issues a
wildcard certificate and displays the certificate and private-key path.
For example:
Your cert is in: /root/.acme.sh/susedojo.com/susedojo.com.cer Your cert key is in: /root/.acme.sh/susedojo.com/susedojo.com.key v2 chain. The intermediate CA cert is in: /root/.acme.sh/susedojo.com/ca.cer And the full chain certs is in: /root/.acme.sh/susedojo.com/fullchain.cer_on_issue_success
Notice the location of your certificate and key. These are now ready to be used by OpenStack Cloud.
To automate the process of setting up the TXT record in your DNS
servers and prepare it for automated validation, the file
account.conf
holds account information
for the DNS Provider. After exporting the authentication variables,
it stores them automatically after the command is executed for later use.
To issue your wildcard certificate, the command without optional settings is:
# export LUA_Key=”your_API_token_from_account” # export LUA_Email=”cameron@yourdomain.com” # acme.sh -d yourdomain.com -d *.yourdomain.com --dns dns_lua
You can now view your DNS records and you will see a new TXT record
available. When it is finished and the DNS validation is okay,
acme.sh
issue your wildcard certificate and displays
your certificate and private-key path just as before.
Your cert is in: /root/.acme.sh/susedojo.com/susedojo.com.cer Your cert key is in: /root/.acme.sh/susedojo.com/susedojo.com.key v2 chain. The intermediate CA cert is in: /root/.acme.sh/susedojo.com/ca.cer And the full chain certs is in: /root/.acme.sh/susedojo.com/fullchain.cer_on_issue_success
Create a shared location for all Certificates on the control nodes. Execute these commands on all control nodes and compute nodes:
mkdir -p /etc/cloud/ssl/certs mkdir -p /etc/cloud/ssl/private
Copy all certificates to their shared locations on the control nodes and compute nodes:
# scp /root/.acme.sh/susedojo.com/susedojo.com.cer \ root@control:/etc/cloud/ssl/certs # scp /root/.acme.sh/susedojo.com/ca.cer root@control:/etc/cloud/ssl/certs # scp /root/.acme.sh/susedojo.com/fullchain.cer root@control:/etc/cloud/ssl/certs # scp /root/.acme.sh/susedojo.com/susedojo.com.key \ root@control:/etc/cloud/ssl/private
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false, and
Require Client Certificates
is true.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false, and
Require Client Certificates
is false.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false, and
Require Client Certificates
is false.
Click Apply
. Your changes will apply in a
few minutes.
Set your Certificate File and Key File to the proper location, and
set the CA Certificates File to the /etc/ssl/ca-bundle.pem
in the distribution. Make sure Generate (self-signed) certificates
is false, and Certificate is insecure
is false, and
Require Client Certificates
is false.
Click Apply
. Your changes will apply in a
few minutes.
Each Crowbar barclamp that have SSL support are the same. You can change the same settings and apply your certificate to the remaining barclamps.
Once this is completed, we recommend automating this process as the
Let's Encrypt
certificates expire after 90 days.
Find a list of log files below, sorted according to the nodes where they can be found.
Crowbar Web Interface:
/var/log/crowbar/production.log
Chef server: /var/log/chef/server.log
Chef expander: /var/log/chef/expander.log
Chef client (for the Administration Server only):
/var/log/chef/client.log
Upgrade log files (only available if the Administration Server has been upgraded
from a previous version using suse-cloud-upgrade
):
/var/log/crowbar/upgrade/*
Apache SOLR (Chef's search server):
/var/log/chef/solr.log
HTTP (AutoYaST) installation server for provisioner barclamp:
/var/log/apache2/provisioner-{access,error}_log
Log file from mirroring SMT repositories (optional):
/var/log/smt/smt-mirror.log
Default SUSE log files: /var/log/messages
,
/var/log/zypper.log
etc.
Syslogs for all nodes: /var/log/nodes/*.log
(these
are collected via remote syslogging)
Other client node log files saved on the Administration Server:
/var/log/crowbar/sledgehammer/d*.log
: Initial
Chef client run on nodes booted using PXE prior to discovery by
Crowbar.
/var/log/crowbar/chef-client/d*.log
: Output from
Chef client when proposals are applied to nodes. This is the
first place to look if a barclamp proposal fails to apply.
Logs for when the node registers with the Administration Server:
/var/log/crowbar/crowbar_join/errlog
/var/log/crowbar/crowbar_join/$TOPIC.{log,err}
:
STDOUT/STDERR from running commands associated with $TOPIC when the
node joins the Crowbar cluster. $TOPIC can be:
zypper
: package management activity
ifup
: network configuration activity
Chef
: Chef client activity
time
: starting of ntp client
Chef client log: /var/log/chef/client.log
Default SUSE log files: /var/log/messages
,
/var/log/zypper.log
etc.
On setups with multiple Control Nodes log files for certain services
(such as keystone.log
) are only available on the
nodes where the services are deployed.
/var/log/apache2/openstack-dashboard-*
: Logs for
the OpenStack Dashboard
/var/log/ceilometer/*
: Ceilometer log files.
/var/log/cinder/*
: Cinder log files.
/var/log/glance/*
: Glance; log files.
/var/log/heat/*
: Heat log files.
/var/log/keystone/*
: Keystone log files.
/var/log/neutron/*
: Neutron log files.
/var/log/nova/*
: various log files relating to
Nova services.
/var/log/rabbitmq/*
: RabbitMQ log files.
/var/log/swift/*
: Swift log files.
/var/log/nova/nova-compute.log
Find solutions for the most common pitfalls and technical details on how to create a support request for SUSE OpenStack Cloud Crowbar here.
If your problem is not mentioned here, checking the log files on either the Administration Server or the OpenStack nodes may help. A list of log files is available at Chapter 19, Log Files.
Check the installation routine's log file at
/var/log/crowbar/install.log
for error messages.
As of SUSE OpenStack Cloud Crowbar 8, it is assumed that each machine can be accessed directly via IPMI/BMC. However, this is not the case on certain blade hardware, where several nodes are accessed via a common adapter. Such a hardware setup causes an error on deploying the IPMI/BMC network. You need to disable the IPMI deployment running the following command:
/opt/dell/bin/json-edit -r -a "attributes.ipmi.bmc_enable" \ -v "false" /opt/dell/chef/data_bags/crowbar/bc-template-ipmi.json
Re-run the SUSE OpenStack Cloud Crowbar installation after having disabled the IPMI deployment.
If route
-n
shows no gateway for
the bastion network, check the value of the following entries in
/etc/crowbar/network.json
:
"router_pref":
and "router_pref":
.
Make sure the value for the bastion network's
"router_pref":
is set to a
lower value than "router_pref":
for the admin network.
If the router preference is set correctly, route
-n
shows a gateway for the bastion network. In case
the Administration Server is still not accessible via its admin network
address (for example,
192.168.124.10
), you need
to disable route verification (rp_filter
). Do so
by running the following command on the Administration Server:
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
If this setting solves the problem, make it permanent by editing
/etc/sysctl.conf
and setting the value for
net.ipv4.conf.all.rp_filter
to
0
.
No, after you have run the SUSE OpenStack Cloud Crowbar installation you cannot change the host name. Services like Crowbar, Chef, and the RabbitMQ will fail after changing the host name.
Tampered with cookie
error?
You probably have an old cookie in your browser from a previous
Chef installation on the same IP address. Remove the cookie named
_chef_server_session_id
and try again.
Custom repositories need to be added using the YaST Crowbar module:
Start the YaST Crowbar module and switch to the
tab: › › › .Choose
Enter the following data:
A unique name to identify the repository.
Link or path to the repository.
Access errors to a repository are silently ignored by default.
To ensure that you get notified of these errors, set the
Ask On Error
flag.
Currently only repositories for
on the architecture are supported. Make sure to select both options.Save your settings selecting
.root
?
By default you cannot directly log in to a node as root
,
because the nodes were set up without a root
password. You
can only log in via SSH from the Administration Server. You should be able to
log in to a node with ssh root@NAME
where NAME
is the name (alias) of the node.
If name resolution does not work, go to the Crowbar Web interface
and open the root
.
Make sure to change the boot order in the BIOS of the node, so that the first boot option is to boot from the network/boot using PXE.
The root
login is enabled at a very early state in discovery
mode, so chances are high that you can log in for debugging purposes as
described in
How can I log in to a node as root?
. If
logging in as root
does not work, you need to set the root
password manually. This can either be done by setting the password via
the Kernel command line as explained in
How to provide Kernel Parameters for the SLEShammer Discovery Image?
, or by creating a hook as
explained below:
Create a directory on the Administration Server named
/updates/discovering-pre
mkdir /updates/discovering-pre
Create a hook script setpw.hook
in the
directory created in the previous step:
cat > /updates/discovering-pre/setpw.hook <<EOF #!/bin/sh echo "root:linux" | chpasswd EOF
Make the script executable:
chmod a+x /updates/discovering-pre/setpw.hook
If you are still cannot log in, you very likely hit a bug in the discovery image. Report it at http://bugzilla.suse.com/.
Kernel Parameters for the SLEShammer Discovery Image can be provided
via the Provisioner barclamp. The following example shows how to set a
root
password:
Open a browser and point it to the Crowbar Web interface available on
the Administration Server, for example
http://192.168.124.10/
. Log in as user crowbar
. The password is
crowbar
by default, if you have not changed it.
Open
› and click in the row.
Click "discovery":
{ "append": "" }
line, for example;
"discovery": { "append": "DISCOVERY_ROOT_PASSWORD=PASSWORD" },
the proposal without changing the assignments in the section.
The installation repository on the
Administration Server at /srv/tftpboot/suse-12.3/install
has not been set up correctly to contain the SUSE Linux Enterprise Server 12 SP3
installation media. Review the instructions at
Section 5.1, “Copying the Product Media Repositories”.
Unpacking
initramfs
during boot when using PXE?
The node probably does not have enough RAM. You need at least 4 GB RAM for the deployment process to work.
Problem
? What to do if a node hangs at
“Executing AutoYast script:
/usr/sbin/crowbar_join --setup” after the installation is finished?
Be patient—the AutoYaST script may take a while to finish. If
it really hangs, log in to the node as root
(see
How can I log in to a node as root?
for details).
Check for error messages at the end of
/var/log/crowbar/crowbar_join/chef.log
. Fix the
errors and restart the AutoYaST script by running the following command:
crowbar_join --start
If successful, the node will be listed in state
Ready
, when the script has finished.
In cases where the initial --setup wasn't able to complete successfully, you can rerun that once after the previous issue is solved.
If that does not help or if the log does not provide useful information, proceed as follows:
Log in to the Administration Server and run the following command:
crowbar crowbar transition $NODE
NODE needs to be replaced by the alias
name you have given to the node when having installed it. Note that
this name needs to be prefixed with $
.
Log in to the node and run chef-client
.
Check the output of the command for failures and error messages and try to fix the cause of these messages.
Reboot the node.
If the node is in a state where login in from the Administration Server is not
possible, you need to create a root
password for it as
described in Direct root
Login.
Now re-install the node by going to the node on the Crowbar Web
interface and clicking . After having
been re-installed, the node will hang again, but now you can log in
and check the log files to find the cause.
Check the Chef client log files on the Administration Server located at
/var/log/crowbar/chef-client/d*.log
. Further
information is available from the Chef client log files located
on the node(s) affected by the proposal
(/var/log/chef/client.log
), and from the log
files of the service that failed to be deployed. Additional
information may be gained from the Crowbar Web UI log files on the
Administration Server. For a list of log file locations refer to
Chapter 19, Log Files.
By default, the OpenStack nodes are installed by booting a discovery image from the Administration Server using PXE. They are allocated and then boot via PXE into an automatic installation (see Section 11.2, “Node Installation” for details). To install the OpenStack nodes manually or with a custom provisioning tool, you need to disable the PXE boot service and the DNS service on the Administration Server.
As a consequence you also need to provide an external DNS server. Such a server needs to comply with the following requirements:
It needs to handle all domain-to-IP requests for SUSE OpenStack Cloud.
It needs to handle all IP-to-domain requests for SUSE OpenStack Cloud.
It needs to forward unknown requests to other DNS servers.
To disable the PXE and DNS services when setting up the Administration Server, proceed as follows:
The following steps need to be performed before starting the SUSE OpenStack Cloud Crowbar installation.
Create the file /etc/crowbar/dns.json
with the
following content:
{ "attributes": { "dns": { "nameservers": [ "DNS_SERVER", "DNS_SERVER2" ], "auto_assign_server": false } } }
Replace DNS_SERVER and DNS_SERVER2 with the IP address(es) of the external DNS server(s). Specifying more than one server is optional.
Create the file /etc/crowbar/provisioner.json
with the following content:
{ "attributes": { "provisioner": { "enable_pxe": false } } }
If these files are present when the SUSE OpenStack Cloud Crowbar installation is started, the Administration Server will be set up using external DNS services and no PXE boot server.
If you already have deployed SUSE OpenStack Cloud, proceed as follows to disable the DNS and PXE services on the Administration Server:
Open a browser and point it to the Crowbar Web interface available on
the Administration Server, for example
http://192.168.124.10/
. Log in as user crowbar
. The password is
crowbar
by default, if you have not changed it.
Open
› and click in the row.
Click false
:
"enable_pxe": false,
the proposal without changing the assignments in the section.
Change to the
barclamp via › and click in the row.
Click false
and
add the address(es) for the external name server(s):
"auto_assign_server": false, "nameservers": [ "DNS_SERVER", "DNS_SERVER2" ],
Replace DNS_SERVER and DNS_SERVER2 with the IP address(es) of the external DNS server(s). Specifying more than one server is optional.
your changes, but do not apply them, yet!
In the
section of the barclamp remove all nodes from the role, but do not change the assignments for the role.the barclamp.
When the DNS barclamp has been successfully applied, log in to the Administration Server and stop the DNS service:
systemctl stop named
Now that the PXE and DNS services are disabled you can install SUSE Linux Enterprise Server 12 SP3 on the OpenStack nodes. When a node is ready, add it to the pool of nodes as described in Section 11.3, “Converting Existing SUSE Linux Enterprise Server 12 SP3 Machines Into SUSE OpenStack Cloud Nodes”.
When adding a new hard disk to a node that has already been deployed, it can take up to 15 minutes before the new disk is detected.
SUSE OpenStack Cloud Crowbar offers the possibility to install additional packages that are not part of the default scope of packages installed on the OpenStack nodes. This is for example required if your hardware is only supported by a third party driver. It is also useful if your setup requires to install additional tools that would otherwise need to be installed manually.
Prerequisite for using this feature is that the packages are available in a repository known on the Administration Server. Refer to How to make custom software repositories from an external server (for example a remote SMT or SUSE M..? for details, if the packages you want to install are not part of the repositories already configured.
To add packages for installation on node deployment, proceed as follows:
Open a browser and point it to the Crowbar Web interface on the Administration Server, for
example http://192.168.124.10/
. Log in as user
crowbar
. The password is
crowbar
by default, if you have not changed it
during the installation.
Go to
› and click the button for .Next click
in the page to open an editable view of the provisioner configuration.Add the following JSON code before the last closing curly bracket (replace the PACKAGE placeholders with real package names):
"packages": { "suse-12.2": ["PACKAGE_1", "PACKAGE_2"], }
Note that these packages will get installed on all OpenStack nodes. If
the change to the Provisioner barclamp is made after nodes have already
been deployed, the packages will be installed on the affected nodes
with the next run of Chef or crowbar-register
. Package
names will be validated against the package naming guidelines to
prevent script-injection.
nova
default configuration?
To change the nova
default configuration, its Chef
cookbook file needs to be adjusted. This file
is stored on the Administration Server at
/opt/dell/chef/cookbooks/nova/templates/default/nova.conf.erb
.
To activate changes to these files, execute the following command:
barclamp_install.rb --rpm /opt/dell/barclamps/openstack/
To change the credentials for the Keystone administrator (admin
) or the regular user (crowbar
by default), proceed as follows:
Log in to the Control Node on which Keystone is deployed as user
root
via the Administration Server.
In a shell, source the OpenStack RC file for the project that you want to upload an image to. For details, refer to Set environment variables using the OpenStack RC file in the OpenStack documentation.
Enter the following command to change the PASSWORD for the administrator or the regular user (USER):
keystone-manage bootstrap --bootstrap-password PASSWORD \ --bootstrap-username USER
For a complete list of command line options, run
keystone-manage bootstrap --help
. Make sure to
start the command with a Space to make sure the
password does not appear in the command history.
Access the Keystone barclamp on the Crowbar Web interface by going to
› and click for the Keystone barclamp.Enter the new password for the same user you specified on the command line before.
Activate the change by clicking
. When the proposal has been re-applied, the password has changed and can be used.
Before contacting support to help you with a problem on SUSE OpenStack Cloud, it is
strongly recommended that you gather as much information about your
system and the problem as possible. For this purpose, SUSE OpenStack Cloud Crowbar
ships with a tool called supportconfig
. It gathers
system information such as the current kernel version being used, the
hardware, RPM database, partitions, and other items.
supportconfig
also collects the most important log
files, making it easier for the supporters to identify and solve your
problem.
It is recommended to always run supportconfig
on the
Administration Server and on the Control Node(s). If a Compute Node or a
Storage Node is part of the problem, run
supportconfig
on the affected node as well. For
details on how to run supportconfig
, see
https://documentation.suse.com/sles/12-SP5/single-html/SLES-admin/#cha-adm-support.
Under certain circumstances, the SUSE support may provide temporary fixes, the so-called PTFs, to customers with an L3 support contract. These PTFs are provided as RPM packages. To make them available on all nodes in SUSE OpenStack Cloud, proceed as follows.
Download the packages from the location provided by the SUSE L3 Support to a temporary location on the Administration Server.
Move the packages from the temporary download location to the following directories on the Administration Server:
*.noarch.rpm
):
/srv/tftpboot/suse-12.3/x86_64/repos/PTF/rpm/noarch/
|
/srv/tftpboot/suse-12.3/s390x/repos/PTF/rpm/noarch/
|
*.x86_64.rpm
)
/srv/tftpboot/suse-12.3/x86_64/repos/PTF/rpm/x86_64/
*.s390x.rpm
)
/srv/tftpboot/suse-12.3/s390x/repos/PTF/rpm/s390x/
Create or update the repository metadata:
createrepo-cloud-ptf
The repositories are now set up and are available for all nodes in SUSE OpenStack Cloud except for the Administration Server. In case the PTF also contains packages to be installed on the Administration Server, make the repository available on the Administration Server as well:
zypper ar -f /srv/tftpboot/suse-12.3/x86_64/repos/PTF PTF
To deploy the updates, proceed as described in
Section 11.4.1, “Deploying Node Updates with the Updater Barclamp”. Alternatively, run
zypper up
manually on each node.
This document will help you to prepare SUSE and prospective customers for a Proof of Concept (PoC) deployment of SUSE OpenStack Cloud. This document provides specific details for a PoC deployment. It serves as an addition to the SUSE OpenStack Cloud Deploying With Crowbar.
This document will help you to prepare SUSE and prospective customers for a Proof of Concept (PoC) deployment of SUSE OpenStack Cloud. This document provides specific details for a PoC deployment. It serves as an addition to the SUSE OpenStack Cloud Deploying With Crowbar.
The latest version 8 of SUSE OpenStack Cloud supports all OpenStack Pike release components for best-in-class capabilities to deploy an open source private cloud. Here is a brief overview of SUSE OpenStack Cloud features and functionality.
Installation Framework Integration with the Crowbar project speeds up and simplifies installation and administration of your physical cloud infrastructure.
Mixed Hypervisor Support Enhanced virtualization management through support for multi-hypervisor environments that use KVM, Xen, VMware vSphere, and IBM z/VM.
High Availability Automated deployment and configuration of control plane clusters. This ensures continuous access to business services and delivery of enterprise-grade Service Level Agreements.
High availability for KVM and Xen Compute Nodes and Workloads Enhanced support for critical workloads not designed for cloud architectures.
Docker Support Gives the ability to build and run innovative containerized applications through Magnum integration.
Scalability Cloud control system designed to grow with your demands.
Open APIs Using the standard APIs, customers can enhance and integrate OpenStack with third-party software.
Block Storage Plug-Ins A wide range of block storage plug-ins available from storage vendors like EMC, NetApp, and others.
Networking Plug-Ins SUSE OpenStack Cloud natively supports open source SDNs via Open vSwitch, harnessing the power of DPDK in SUSE Linux Enterprise Server 12 SP3. For more flexibility, SUSE OpenStack Cloud provides support for third-party tools from Cisco, Midokura, Infoblox, Nuage Networks, PLUMgrid and even VLAN bridging solutions.
Award-Winning Support SUSE OpenStack Cloud is backed by 24x7 worldwide-technical support.
Full Integration with SUSE Update Processes Easily maintain and patch cloud deployments.
Non-Disruptive Upgrade Capabilities Simplify migration to future SUSE OpenStack Cloud releases.
The following is a brief overview of components for setting up and managing SUSE OpenStack Cloud.
Administration Server provides all services needed to manage and deploy all other nodes in the cloud. Most of these services are provided by the Crowbar tool. Together with Chef, Crowbar automates all the required installation and configuration tasks. The services provided by the server include DHCP, DNS, NTP, PXE, TFTP.
The Administration Server also hosts the software repositories for SUSE Linux Enterprise Server and SUSE OpenStack Cloud. These repositories are required for node deployment. If no other sources for the software repositories are available, the Administration Server can also host the Subscription Management Tool (SMT), providing up-to-date repositories with updates and patches for all nodes.
Control Nodes host all OpenStack services for orchestrating virtual machines deployed on the compute nodes. OpenStack in SUSE OpenStack Cloud uses a PostgreSQL database that is also hosted on the Control Nodes. When deployed, the following OpenStack components run on the Control Nodes:
MariaDB
Image (Glance)
Identity (Keystone)
Networking (Neutron)
Block Storage (Cinder)
Shared Storage (Manila)
OpenStack Dashboard
Keystone
Pacemaker
Nova controller
Message broker
Swift proxy server
Hawk monitor
Heat an orchestration engine
Ceilometer server and agents
Trove a Database-as-a-Service
A single Control Node running multiple services can become a performance bottleneck, especially in large SUSE OpenStack Cloud deployments. It is possible to distribute the services listed above on more than one Control Node. This includes scenarios where each service runs on its own node.
Compute Nodes are a pool of machines for running instances. These machines require an adequate number of CPUs and enough RAM to start several instances. They also need to provide sufficient storage. A Control Node distributes instances within the pool of compute nodes and provides the necessary network resources. The OpenStack service Compute (Nova) runs on Compute Nodes and provides means for setting up, starting, and stopping virtual machines. SUSE OpenStack Cloud supports several hypervisors such as KVM, VMware vSphere, and Xen. Each image that can be started with an instance is bound to one hypervisor. Each Compute Node can only run one hypervisor at a time. For a PoC deployment, SUSE recommends to leverage KVM as hypervisor of choice.
Optional Storage Nodes Storage Node is a pool of machines that provide object or block storage. Object storage supports several back-ends.
Although each customer has a specific set of requirements, it is important to have 3-5 clearly-defined objectives. This objectives should be provable, measurable and have a specific time scale in which proof is required. The objectives can be adjusted and amended, provided that both parties are agreed on the changes. For a full record of the performed and completed work, it is recommended to use this document for making amendments to the proof requirements.
Before deploying SUSE OpenStack Cloud, it is necessary to meet certain requirements and consider various aspects of the deployment. Some decisions need to be made before deploying SUSE OpenStack Cloud, since they cannot be changed afterward.
The following procedure covers preparatory steps for the deployment of SUSE OpenStack Cloud along with the software and hardware components required for a successful implementation.
Make sure that the required hardware and virtual machines are provided and configured
Check that PXE boot from the first NIC in BIOS is enabled
Ensure that the hardware is certified for use with SUSE Linux Enterprise Server 12 SP3
Check that booting from ISO images works
Make sure that all NICs are visible
Install sar/sysstat
for performance troubleshooting
Ensure that all needed subscription records are available. Depending on the size of the cloud to be implemented, this includes the following:
SUSE OpenStack Cloud subscriptions
SUSE Linux Enterprise Server subscriptions
SLES High Availability Extensions (HAE) subscriptions
Optional SUSE Enterprise Storage (SES) subscriptions
Check whether all needed channels and updates are available either locally or remotely. The following options can be used to provide the repositories and channels:
SMT server on the administration server (optional step)
Existing SMT server
Existing SUSE Manager
Make sure that networking planed and wired according to the specified layout or topology
If SUSE Enterprise Storage is a part of the PoC deployment, all nodes must be installed, configured, and optimized before installing SUSE OpenStack Cloud. Storage services (Nova, Cinder, Glance, Cluster STONITH,) required by SUSE OpenStack Cloud 6 must be available and accessible.
Check whether network.json
is configured
according to the specific requirements. This step must be discussed
and completed in advance.
The hardware and software matrix below has the following requirements:
All machines must run SUSE Linux Enterprise Server 12 SP3
KVM or Xen Hypervisors must be running on bare metal
The Admin Node can be deployed on a KVM or VMware virtual machine
The sizing recommendation includes an Admin Node (bare metal or VM), Controller Nodes, Compute Nodes to host all your OpenStack services, and the optional SES Nodes. The matrix also provides information on the necessary network equipment and bandwidth requirements.
These recommendations are based on real-world use cases and experience gathered by SUSE in the last three years. However, these recommendations are meant to serve as guidelines and not as requirements. The final sizing decision depends on the actual customer workloads and architecture, which must be discussed in depth. The type and number of hardware components such as hard disks, CPU, and RAM also serve as starting points for further discussion and evaluation depending on workloads.
Number of Units | Function | Configuration | OpenStack Component |
---|---|---|---|
3 | Compute nodes | 2 hard disks 2 Quad Core Intel or AMD processors 256GB RAM 2 or 4 10Gb Ethernet NICs | Nova-multi-compute ML2 Agent OVS Agent |
1 | Admin Node or VM | 2 hard disks 1 Quad Core Intel or AMD processor 8GB RAM 2 or 4 10Gb Ethernet NICs | Crowbar, tftpboot, PXE |
3 | Control node | 2 hard disks 2 Quad Core Intel or AMD processors 2x64GB RAM 2 or 4 10Gb Ethernet NICs | Horizon Rabbit MQ Nova multi-controller Cinder Glance Heat Ceilometer Neutron-Server ML2 Agent Keystone MariaDB Neutron ML2 Plugin L2/L3 Agents DHCP Agent |
CloudFoundry | 48 vCPUs 256GB RAM Min 2TB Storage | SUSE Cloud Application Platform | |
4 | Storage Server – SUSE Enterprise Storage | 2 hard disks 2 Quad Core Intel or AMD processors 64GB RAM 2 or 4 10Gb Ethernet NICs | Admin – Server MON - Server OSD - Server |
1 | Switch min. 10 GbE ports | All VLANs/Tagged or Untagged | |
OS: SUSE Linux Enterprise Server 12 SP3 | |||
DHCP, DNS | Isolated within administrator network | ||
HA | 3 Control Nodes |
Configuring and managing your network are two of the most challenging tasks of deploying a SUSE OpenStack Cloud. Therefore they need to be planned carefully. Just as OpenStack provides flexibility and agility for compute and storage, SDN in OpenStack gives cloud administrators more control over their networks. However, building and manually configuring the virtual network infrastructure for OpenStack is difficult and error-prone. SUSE OpenStack Cloud solve this by delivering a structured installation process for OpenStack which could be customized to adapt the given environment.
The deployment of the network configuration is done while setting up an Administrator Node. As a requirement for the deployment, the entire network configuration needs to be specified in the network.json
file.
The Crowbar network barclamp provides two functions for the system:
Initialization of network interfaces on the Crowbar managed systems
Address pool management. While the addresses can be managed with the YaST Crowbar module, complex network setups require to manually edit the network barclamp template file /etc/crowbar/network.json
. For more detailed explanation and description see https://documentation.suse.com/soc/8/single-html/suse-openstack-cloud-deployment/#sec-deploy-network-json-edit.
The network definitions contain IP address assignments, the bridge and VLAN setup, and settings for the router preference. Each network is also assigned to a logical interface. These VLAN IDs and networks can be modified according to the customer's environment.
SUSE OpenStack Cloud supports three network modes: single, dual and team. As of SUSE OpenStack Cloud 6, the network mode is applied to all nodes and the Administration Server. That means that all machines need to meet the hardware requirements for the chosen mode. The following network modes are available:
Single Network Mode In single mode one Ethernet card is used for all the traffic.
Dual Network Mode Dual mode needs two Ethernet cards (on all nodes but Administration Server). This allows to completely separate traffic to and from the administrator network and to and from the public network.
Team Network Mode The team mode is almost identical to single mode, except it combines several Ethernet cards to a so-called bond (network device bonding). Team mode requires two or more Ethernet cards.
In an HA configuration, make sure that SUSE OpenStack Cloud is deployed with the team network mode.
The following networks are pre-defined for use with SUSE OpenStack Cloud.
Network Name | VLAN | IP Range |
---|---|---|
Router | No - untagged | 192.168.124.1 |
Admin | No - untagged | 192.168.124.10 – 192.168.124.11 |
DHCP | No - untagged | 192.168.124.21 – 192.168.124.80 |
Host | No - untagged | 192.168.124.81 – 192.168.124.160 |
BMC VLAN Host | 100 | 192.168.124.61 |
BMC Host | No - untagged | 192.168.124.162 – 192.168.124.240 |
Switch | No - untagged | 192.168.124.241 – 192.168.124.250 |
Network Name | VLAN | IP Range |
---|---|---|
Router | 500 | 192.168.123.1 – 192.168.123.49 |
DHCP | 500 | 192.168.123.50 – 192.168.123.254 |
Network Name | VLAN | IP Range |
---|---|---|
Public Host | 300 | 192.168.126.2 – 192.168.126.49 |
Public DHCP | 300 | 192.168.126.50 – 192.168.126.127 |
Floating Host | 300 | 192.168.126.129 – 192.168.126.191 |
Network Name | VLAN | IP Range |
---|---|---|
Host | 200 | 192.168.125.2 – 192.168.125.254 |
The default IP addresses can be changed using YaST Crowbar module or by editing the appropriate JSON file. It is also possible to customize the network setup for your environment. This can be done by editing the network barclamp template.
SUSE OpenStack Cloud requires a complex network setup consisting of several networks configured during installation. These networks are reserved for cloud usage. Access to these networks from an existing network requires a router.
The network configuration on the nodes in the SUSE OpenStack Cloud network is controlled by Crowbar. Any network configuration changes done outside Crowbar will be automatically overwritten. After the cloud is deployed, network settings cannot be changed without reinstalling the cloud.
Controller Node serves as the front-end for API calls to the compute, image, volume, network, and orchestration services. In addition to that, the node hosts multiple Neutron plug-ins and agents. The node also aggregates all route traffic within tenant and between tenant network and outside world.
Compute Node creates on-demand virtual machines using chosen hypervisor for customer application.
Administrator Node automates the installation processes via Crowbar using pre-defined cookbooks for configuring and deploying a Control Node and Compute and Network Nodes.
The Administrator Node requires a dedicated local and isolated DHCP/PXE environment controlled by Crowbar.
Optional storage access. Cinder is used for block storage access exposed through iSCSI or NFS (FC connection is not supported in the current release of OpenStack). This could also be a dedicated Storage Node.
Implementation of Ceph as a cloud storage requires a high-performance network. The Storage Net in the following network topology would allow the respective OpenStack services to connect to the external Ceph storage cluster public network. It is recommended to run a Ceph storage cluster with two networks: a public network and a cluster network. To support two networks, each Ceph node must have more than one NIC.
Network mode. What mode to choose for a PoC deployment depends on the High Availability (HA) requirements. The team network mode is required for an HA setup of SUSE OpenStack Cloud.
VLAN support for the administrator network must be handled at the switch level. The following networks are predefined when setting up SUSE OpenStack Cloud. The listed default IP addresses can be changed using the YaST Crowbar module. It is also possible to customize the network setup.
The default network proposal described below allows maximum 80 Compute Nodes, 61 floating IP addresses, and 204 addresses in the nova_fixed network. To overcome these limitations, you need to reconfigure the network setup by using appropriate address ranges manually.
A private network to access the Administration Server and all nodes for administration purposes. The default setup lets you also access the Baseboard Management Controller (BMC) data via Intelligent Platform Management Interface (IPMI) from this network. If required, BMC access can be swapped to a separate network.
You have the following options for controlling access to this network:
Do not allow access from the outside and keep the administrator network completely separated
Allow access to the administration server from a single network (for example, your company's administration network) via the “bastion network” option configured on an additional network card with a fixed IP address
Allow access from one or more networks via a gateway
Private SUSE OpenStack Cloud internal virtual network. This network is used by Ceph and Swift only. It should not be accessed by users.
Private SUSE OpenStack Cloud internal virtual network. This network is used for communication between instances and provides them with access to the outside world. SUSE OpenStack Cloud automatically provides the required gateway.
The only public network provided by SUSE OpenStack Cloud. On this network, you can access the Nova Dashboard and all instances (provided they are equipped with a floating IP). This network can only be accessed via a gateway that needs to be provided externally. All SUSE OpenStack Cloud users and administrators need to be able to access the public network.
Private SUSE OpenStack Cloud internal virtual network. This network is used when Neutron is configured to use openvswitch
with GRE tunneling for the virtual networks. It should not be accessed by users.
SUSE OpenStack Cloud supports different network modes: single, dual, and team. Starting with SUSE OpenStack Cloud 6, the networking mode is applied to all nodes and the Administration Server. This means that all machines need to meet the hardware requirements for the chosen mode. The network mode can be configured using the YaST Crowbar module (see https://documentation.suse.com/soc/8/single-html/suse-openstack-cloud-deployment/#sec-depl-adm-inst-crowbar). The network mode cannot be changed after the cloud is deployed.
More flexible network mode setups can be configured by editing the Crowbar network configuration files (see https://documentation.suse.com/soc/8/single-html/suse-openstack-cloud-deployment/#sec-deploy-network-json-edit for more information). SUSE or a partner can assist you in creating a custom setup within the scope of a consulting services agreement. For more information on SUSE consulting, visit http://www.suse.com/consulting/.
Team network mode is required for an HA setup of SUSE OpenStack Cloud. If you are planning to move your cloud to an HA setup later, deploy SUSE OpenStack Cloud with team network mode right from the beginning. Migration to an HA setup is not supported.
SUSE OpenStack Cloud is based on SUSE Linux Enterprise Server 12 SP3, OpenStack, Crowbar and Chef. SUSE Linux Enterprise Server is used as the underlying operating system for all infrastructure nodes. Crowbar and Chef are used to automatically deploy and manage the OpenStack nodes from a central Administration Server.
After you have successfully deployed OpenStack, you need to test the environment by using either the Dashboard or the command line interface. This document provides the most important procedures and steps to perform functional tests agreed upon. A detailed list of test case should be provided with this document.
All test cases are work in progress and by no means complete. Test cases have to be formulated by the entire team according to the requirements and type of workloads.
Add your own test cases here.
Add your own test cases here.
SUSE OpenStack Cloud supports the Nova Compute VMware vCenter driver. It enables access to advanced features such as vMotion, High Availability, and Dynamic Resource Scheduling (DRS). However, VMware vSphere is not supported “natively” by SUSE OpenStack Cloud—it rather delegates requests to an existing vCenter. It requires preparations at the vCenter and post install adjustments of the Compute Node.
The following requirements must be met to successfully deploy a Nova Compute VMware node:
VMware vSphere vCenter 6.0 or higher
VMware vSphere ESXi nodes 6.0 or higher
A separate Compute Node that acts as a proxy to vCenter is required. Minimum system requirements for this node are:
CPU: x86_64 with 2 cores (4 recommended) |
RAM: 2 GB (8 GB recommended) |
Disk space: 4 GB (30 GB recommended) |
See Section A.3, “Finishing the Nova Compute VMware Node Installation” for setup instructions.
Neutron must not be deployed with the openvswitch with
gre
plug-in, a VLAN setup is required.
SUSE OpenStack Cloud requires the VMware vCenter server to run version 5.1 or better. You need to create a single data center for SUSE OpenStack Cloud (multiple data centers are currently not supported):
Log in to the vCenter Server using the vSphere Web Client
Choose
and create a singleSet up a
which has enabled.
Set Fully
Automated
and to
Aggressive
.
Create shared storage. Only shared storage is supported and data stores must be shared among all hosts in a cluster. It is recommended to remove data stores not intended for OpenStack from clusters being configured for OpenStack. Multiple data stores can be used per cluster.
Create a port group with the same name as the
vmware.integration_bridge
value in
nova.conf
(default is br-int). All VM NICs are
attached to this port group for management by the OpenStack
networking plug-in. Assign the same VLAN ID as for the neutron network.
On the default network setup this is the same VLAN ID as for the
nova_fixed
network. Use
› › › to look up the VLAN ID.
Deploy Nova as described in Section 12.11, “Deploying Nova” on a single Compute Node and fill in the attributes:
IP address of the vCenter server.
vCenter login credentials.
A comma-separated list of cluster names you have added on the vCenter server.
Regular expression to match the name of a data store. If you have
several data stores, this option allows you to specify the
data stores to use with Nova Compute. For example, the value nas.*
selects
all data stores that have a name starting with nas
. If
this option is omitted, Nova Compute uses the first data store returned by the vSphere
API. However, it is recommended not to use this option and to remove data
stores that are not intended for OpenStack instead.
The physical interface that is to be used for VLAN networking. The
default value of vmnic0
references the first
available interface (“eth0”). vmnic1
would be the second interface (“eth1”).
Absolute path to the vCenter CA certificate.
Default value: false
(the CA truststore is used for verification).
Set this option to true
when using self-signed certificates to disable
certificate checks. This setting is for testing purposes only and must not be used in
production environments!
OpenStack does not support deploying multiple VMware Compute Nodes. As
a workaround, set up an instance on the vSphere Cluster, register it
with Crowbar and deploy the
nova-compute-vmware
role on this node:
Create an instance on the vSphere Cluster and install SUSE Linux Enterprise Server 12 SP3.
Configure a network interface in a way that it can access the SUSE OpenStack Cloud admin network.
Enable the High-Availability flag in vCenter for this instance.
Follow the instructions at Section 11.3, “Converting Existing SUSE Linux Enterprise Server 12 SP3 Machines Into SUSE OpenStack Cloud Nodes” to register the instance with the Administration Server and add it to the pool of nodes available for deployment.
Deploy the nova-compute-vmware
role on the new
node as described in Section 12.11, “Deploying Nova” and
Section A.3, “Finishing the Nova Compute VMware Node Installation”.
The following requirements must be met to use Cisco Nexus switches with Neutron:
Cisco Nexus series 3000, 5000 or 7000
All Compute Nodes must be equipped with at least two network cards.
The switch needs to have the XML management interface enabled. SSH access to the management interface must be enabled (refer to the switch's documentation for details).
Enable VLAN trunking for all Neutron managed VLANs on the switch port to which the controller node running Neutron is connected to.
Before deploying Neutron, check if VLAN configurations for Neutron managed VLANs already exist on the switch (for example, from a previous SUSE OpenStack Cloud deployment). If yes, delete them via the switch's management interface prior to deploying Neutron.
When using the Cisco plugin, Neutron reconfigures the VLAN trunk
configuration on all ports used for the nova-fixed
traffic (the traffic between the instances). This requires to
configure separate network interfaces exclusively used by
nova-fixed
. This can be achieved by adjusting
/etc/crowbar/network.json
(refer to
Section 7.5, “Custom Network Configuration”). The following example shows
an appropriate configuration for dual mode, where
has been mapped to conduit
and all other networks to other conduits.
Configuration attributes not relevant in this context have been
replaced with ...
.
{ "attributes" : { "network" : { "conduit_map" : [ ... ], "mode" : "single", "networks" : { "nova_fixed" : { ..., "conduit" : "intf1" }, "nova_floating" : { ..., "conduit" : "intf0" }, "public" : { ..., "conduit" : "intf0" }, "storage" : { ..., "conduit" : "intf0" }, "os_sdn" : { ..., "conduit" : "intf0" }, "admin" : { ..., "conduit" : "intf0" }, "bmc" : { ..., "conduit" : "bmc" }, "bmc_vlan" : { ..., "conduit" : "intf2" }, }, ..., }, } }
Make a note of all switch ports to which the interfaces using the
nova-fixed
network on the Compute Nodes are
connected. This information will be needed when deploying Neutron.
Create a Neutron barclamp proposal in the Crowbar Web interface.
As the ml2
.
As cisco_nexus
.
In vlan
.
In the
table, enter the , the SSH number and the login credentials for the switch's management interface. If you have multiple switches, open a new row in the table by clicking and enter the data for another switch.Choose whether to encrypt public communication (SSL Support: Protocol for configuration details.
) or not ( ). If choosing , refer toChoose a node for deployment and
the proposal.Deploy Nova (see Section 12.11, “Deploying Nova”), Horizon (see Section 12.12, “Deploying Horizon (OpenStack Dashboard)” and all other remaining barclamps.
When all barclamps have been deployed, return to the Neutron barclamp by choosing
› › › . The proposal now contains an additional table named , listing all Compute Nodes.
For each Compute Node enter the switch it is connected to and the port
number from the notes you took earlier. The values need to be entered
like the following: 1/13
or
Eth1/20
.
When you have entered the data for all Compute Nodes, re-apply the proposal.
Whenever you deploy additional Compute Nodes to an active SUSE OpenStack Cloud deployment using the Cisco plugin with Neutron, update the Neutron barclamp proposal by entering their port data as described in the previous step.
To verify if Neutron was correctly deployed, do the following:
Launch an instance (refer to the End User Guide, chapter Launch and manage instances for instructions).
Find out which VLAN was assigned to the network by running the command
neutron net-show fixed
. The result lists a
matching the VLAN.
Log in to the switch's management interface and list the VLAN configuration. If the setup was deployed correctly, the port of the Compute Node the instance is running on, is in trunk mode for the matching VLAN.
This chapter lists content changes for this document since the release of SUSE® OpenStack Cloud Crowbar 8.0.
This manual was updated on the following dates:
In Section 12.8, “Deploying Glance”, corrected the name of an example Nova configuration file for custom settings (https://bugzilla.suse.com/show_bug.cgi?id=1077947).
In Section 12.8, “Deploying Glance”, updated the entries of a drop-down box (https://bugzilla.suse.com/show_bug.cgi?id=1073333).
Numerous small fixes and corrections throughout the document (http://bugzilla.suse.com/show_bug.cgi?id=1073508, https://bugzilla.suse.com/show_bug.cgi?id=1073516).
A concept of how services are running on nodes in a High Availability cluster. In an active/active setup, both the main and redundant systems are managed concurrently. If a failure of services occurs, the redundant system is already online, and can take over until the main system is fixed and brought back online.
A concept of how services are running on nodes in a High Availability cluster. In an active/passive setup, one or more services are running on an active cluster node, whereas the passive node stands by. If the active node fails then the services are transferred to the passive node.
Also called Crowbar Administration Node. Manages all other nodes. It assigns IP addresses to them, boots them using PXE, configures them, and provides them the necessary software for their roles. To provide these services, the Administration Server runs Crowbar, Chef, DHCP, TFTP, NTP, and other services.
A virtual machine that can be created and customized by a user. AMIs can
be identified by an ID prefixed with ami-
.
An OpenStack method of partitioning clouds. It enables you to arrange OpenStack Compute hosts into logical groups. The groups typically have physical isolation and redundancy from other availability zones, for example, by using separate power supply or network equipment for each zone. When users provision resources, they can specify from which availability zone their instance should be created. This allows cloud consumers to ensure that their application resources are spread across disparate machines to achieve high availability if the hardware fails. Since the Grizzly release, availability zones are implemented via host aggregates.
A collection of remote computing services (including Amazon EC2, Amazon S3, and others) that together make up Amazon's cloud computing platform.
A set of Chef cookbooks, templates, and other logic. Used to apply a particular Chef role to individual nodes or a set of nodes.
Code name for Telemetry.
Cells provide a new way to scale Compute deployments. This includes the ability to have compute clusters (cells) in different geographic locations all under the same Compute API. This allows for a single API server being used to control access to multiple cloud installations. Cells provide logical partitioning of Compute resources in a child/parent relationship.
A massively scalable, open source, distributed storage system. It consists of an object store, a block store, and a POSIX-compliant distributed file system.
An automated configuration management platform for deployment of your entire cloud infrastructure. The Chef server manages many of the software packages and allows the easy changing of nodes.
Code name for OpenStack Block Storage.
A package commonly installed in virtual machine images. It uses the SSH public key to initialize an instance after boot.
A set of connected computers that work together. In many respects (and from the outside) they can be viewed as a single system. Clusters can be further categorized depending on their purpose, for example: High Availability clusters, high-performance clusters, or load-balancing clusters.
Whenever communication fails between one or more nodes and the rest of the cluster, a cluster partition occurs: The nodes of a cluster are split into partitions but still active. They can only communicate with nodes in the same partition and are unaware of the separated nodes. As the loss of the nodes on the other partition cannot be confirmed, a Split Brain scenario develops.
The main management entity in a High Availability cluster responsible for coordinating all non-local interactions. The SUSE Linux Enterprise High Availability Extension uses Pacemaker as CRM. Each node of the cluster has its own CRM instance. The instance running on the Designated Coordinator (DC) is the one elected to relay decisions to the other non-local CRMs and to process their input.
Node within a SUSE OpenStack Cloud. A physical server running a Hypervisor. A
Compute Node is a host for guest virtual machines that are deployed in
the cloud. It starts virtual machines on demand using
nova-compute
. To split virtual machine load across
more than one server, a cloud should contain multiple Compute Nodes.
A container is a storage compartment for data. It can be thought of as a directory, only that it cannot be nested.
Node within a SUSE OpenStack Cloud. The Control Node is configured through the
Administration Server and registers with the Administration Server for all required
software. Hosts the OpenStack API endpoints and the OpenStack
scheduler and runs the nova
services—except
for nova-compute
, which is run on the Compute Nodes.
The Control Node coordinates everything about cloud virtual machines:
like a central communication center it receives all requests (for
example, if a user wants to start or stop a virtual machine). It
communicates with the Compute Nodes to coordinate fulfillment of the
request. A cloud can contain multiple Control Nodes.
A collection of Chef recipes which deploy a software stack or functionality. The unit of distribution for Chef.
The messaging/infrastructure layer used in a High Availability cluster that is set
up with SUSE Linux Enterprise High Availability Extension. For example, the cluster communication
channels are defined in
/etc/corosync/corosync.conf
.
Bare-metal installer and an extension of Chef server. The primary function of Crowbar is to get new hardware into a state where it can be managed by Chef. That means: Setting up BIOS and RAID, network, installing a basic operating system, and setting up services like DNS, NTP, and DHCP. The Crowbar server manages all nodes, supplying configuration of hardware and software.
One Cluster Resource Manager in a High Availability cluster is elected as the Designated Coordinator (DC). The DC is the only entity in the cluster that can decide that a cluster-wide change needs to be performed. For example, fencing a node or moving resources around. After a membership change, the DC is elected from all nodes in the cluster.
DRBD is a block device designed for building high availability clusters. The whole block device is mirrored via a dedicated network and is seen as a network RAID-1.
Block-level storage volumes for use with Amazon EC2 instances. Similar to OpenStack Cinder.
A public cloud run by Amazon. It provides similar functionality to OpenStack Compute.
Ephemeral disks offer machine local disk storage linked to the life cycle of a virtual machine instance. When a virtual machine is terminated, all data on the ephemeral disk is lost. Ephemeral disks are not included in any snapshots.
Occurs when a resource fails on a cluster node (or the node itself fails) and the affected resources are started on another node.
Describes the concept of preventing access to a shared resource by isolated or failing cluster members. Should a cluster node fail, it will be shut down or reset to prevent it from causing trouble. The resources running on the cluster node will be moved away to another node. This way, resources are locked out of a node whose status is uncertain.
When an instance is launched, it is automatically assigned a fixed (private) IP address, which stays the same until the instance is explicitly terminated. Private IP addresses are used for communication between instances.
The compute, memory, and storage capacity of nova
computing instances (in terms of virtual CPUs, RAM, etc.). Flavors can
be thought of as “templates” for the amount of cloud
resources that are assigned to an instance.
An IP address that a Compute project can associate with a virtual machine. A pool of floating IP addresses is available in OpenStack Compute, as configured by the cloud operator. After a floating IP address has been assigned to an instance, the instance can be reached from outside the cloud by this public IP address. Floating IP addresses can be dynamically disassociated and associated with other instances.
Code name for OpenStack Image.
An instance of an operating system installed on a virtual machine.
Code name for Orchestration.
High Availability clusters seek to minimize two things: system downtime and data loss. System downtime occurs when a user-facing service is unavailable beyond a specified maximum amount of time. System downtime and data loss (data is accidentally destroyed) can occur not only in case of a single failure. There are also cases of cascading failures, where a single failure deteriorates into a series of consequential failures.
Code name for OpenStack Dashboard.
A physical computer.
An OpenStack method of grouping hosts via a common set of metadata. It enables you to tag groups of hosts with certain capabilities or characteristics. A characteristic could be related to physical location, allowing creation or further partitioning of availability zones. It could also be related to performance (for example, indicating the availability of SSD storage) or anything else that the cloud administrators deem appropriate. A host can be in more than one host aggregate.
One of several deployment models for a cloud infrastructure. A composition of both public and private clouds that remain unique entities, but are bound together by standardized technology for enabling data and application portability. Integrating SUSE Studio and SUSE Manager with SUSE OpenStack Cloud delivers a platform and tools with which to enable enterprise hybrid clouds.
A piece of computer software, firmware or hardware that creates and runs virtual machines. It arbitrates and controls access of the virtual machines to the underlying hardware.
A service model of cloud computing where processing, storage, networks, and other fundamental computing resources are rented over the Internet. It allows the customer to deploy and run arbitrary software, including operating systems and applications. The customer has control over operating systems, storage, and deployed applications but does not control the underlying cloud infrastructure. Housing and maintaining it is in the responsibility of the service provider.
A file that contains a complete Linux virtual machine.
In the SUSE OpenStack Cloud Crowbar context, images are virtual disk images that represent the contents and structure of a storage medium or device (such as a hard disk), in a single file. Images are used as a template from which a virtual machine can be started. For starting a virtual machine, SUSE OpenStack Cloud Crowbar always uses a copy of the image.
Images have both content and metadata; the latter are also called image properties.
A virtual machine that runs inside the cloud.
A point-in-time copy of an instance. It preserves the disk state of a running instance and can be used to launch a new instance or to create a new image based upon the snapshot.
OpenStack Compute injects SSH keypair credentials that are injected into images when they are launched.
Code name for OpenStack Identity.
Virtualization API library. Used by OpenStack to interact with many of its supported hypervisors.
A software allowing multiple virtual machines to share a single physical NIC within OpenStack Compute. It behaves like a hub: You can connect multiple (physical or virtual) network interface devices to it. Any Ethernet frames that come in from one interface attached to the bridge is transmitted to all other devices.
Acts as a virtual disk partition. After creating a Volume Group (VG), logical volumes can be created in that volume group. Logical volumes can be used as raw block devices, swap devices, or for creating a (mountable) file system like disk partitions.
The process of moving a virtual machine instance from one Compute Node to another. This process can only be executed by cloud administrators.
A technology used for a one-to-many communication within a network that can be used for cluster communication. Corosync supports both multicast and unicast.
In the OpenStack Networking API: An isolated L2 network segment (similar to a VLAN). It forms the basis for describing the L2 network topology in a given OpenStack Networking deployment.
Code name for OpenStack Networking.
A (physical) server that is managed by Crowbar.
Code name for OpenStack Compute.
Basic storage entity in OpenStack Object Storage, representing a file that your store there. When you upload data to OpenStack Object Storage, the data is neither compressed nor encrypted, it is stored as-is.
A virtual networking device. It behaves like a virtual switch: network interface devices connect to its ports. The ports can be configured similar to a physical switch's port, including VLAN configurations.
A collection of open source software to build and manage public and private clouds. Its components are designed to work together to provide Infrastructure as a Service and massively scalable cloud computing software.
At the same time, OpenStack is also a community and a project.
One of the core OpenStack components and services (code name:
Cinder
). It provides persistent block
level storage devices for use OpenStack compute instances. The block
storage system manages the creation, attaching and detaching of the
block devices to servers.
Prior to the OpenStack Grizzly release, the service was part of
nova-volume
(block service).
One of the core OpenStack components and services (code name:
Nova
). It is a cloud computing fabric
controller and as such, the main part of an IaaS system. It provides
virtual machines on demand.
One of the core OpenStack components or services (code name:
Horizon
). It provides a modular Web interface for
OpenStack services and allows end users and administrators to interact
with each OpenStack service through the service's API.
One of the core OpenStack components or services (code name:
Keystone
). It provides authentication and
authorization for all OpenStack services.
One of the core OpenStack components or services (code name:
Glance
). It provides discovery, registration, and
delivery services for virtual disk images.
One of the core OpenStack components or services (code name:
Neutron
). It provides “network connectivity
as a service” between interface devices (for example, vNICs)
managed by other OpenStack services (for example, Compute). Allows
users to create their own networks and attach interfaces to them.
One of the core OpenStack components or services (code name:
Swift
). Allows to store and retrieve files
while providing built-in redundancy and fail-over. Can be used for
backing up and archiving data, streaming data to a user's Web browser,
or developing new applications with data storage integration.
A collection of Linux services (or daemons) that work together to provide core functionality within the OpenStack project. This can be storing objects, providing virtual servers, or authentication and authorization. All services have code names, which are also used in configuration files, and command line programs.
A module (code name: Heat
) to orchestrate
multiple composite cloud applications using file-based or Web-based
templates. It contains both a user interface and an API and describes
your cloud deployment in a declarative language. The module is an
integrated project of OpenStack as of the Havana release.
A service model of cloud computing where a computing platform and cloud-based application development tools are rented over the Internet. The customer controls software deployment and configuration settings, but not the underlying cloud infrastructure including network, servers, operating systems, or storage.
An open source cluster resource manager used in SUSE Linux Enterprise High Availability Extension.
In the OpenStack Networking API: An attachment port to an L2 OpenStack Networking network.
One of several deployment models for a cloud infrastructure. The infrastructure is operated exclusively for a single organization and may exist on or off premises. The cloud is owned and managed by the organization itself, by a third party or a combination of both.
See Fixed IP Address.
A concept in OpenStack Identity. Used to identify a group, an
organization, or a project (or more generically, an individual customer
environment in the cloud). Also called tenant
. The
term tenant
is primarily used in the OpenStack
command line tools.
Special configuration for a barclamp. It includes barclamp-specific settings, and a list of nodes to which the proposal should be applied.
One of several deployment models for a cloud infrastructure. The cloud infrastructure is designed for use by the general public and exists on the premises of the cloud provider. Services like applications, storage, and other resources are made available to the general public for free or are offered on a pay-per-use model. The infrastructure is owned and managed by a business, academic or government organization, or some combination of these.
See Floating IP Address.
A disk image format supported by the QEMU virtual machine manager. A
qcow2
image helps to optimize disk space. It
consumes disk space only when contents are written on it and grows as
data is added.
qcow2
is a more recent version of the
qcow
format where a read-only base image is used, and
all writes are stored to the qcow2
image.
In a cluster, a Cluster Partition is defined to have quorum (is “quorate”) if it has the majority of nodes (or votes). Quorum distinguishes exactly one partition. It is part of the algorithm to prevent several disconnected partitions or nodes from proceeding and causing data and service corruption (Split Brain). Quorum is a prerequisite for Fencing, which then ensures that quorum is indeed unique.
Restriction of resources to prevent overconsumption within a cloud. In OpenStack, quotas are defined per project and contain multiple parameters, such as amount of RAM, number of instances, or number of floating IP addresses.
Environment file needed for the OpenStack command line tools. The RC file is project-specific and contains the credentials used by OpenStack Compute, Image, and Identity services.
A group of Chef scripts and templates. Recipes are used by Chef to deploy a unit of functionality.
An OpenStack method of aggregating clouds. Regions are a robust way to share some infrastructure between OpenStack compute installations, while allowing for a high degree of failure tolerance. Regions have a separate API endpoint per installation.
In a High Availability context: Any type of service or application that is known to the cluster resource manager. Examples include an IP address, a file system, or a database.
A script acting as a proxy to manage a resource in a High Availability cluster. For example, it can start, stop or monitor a resource.
In the Crowbar/Chef context: an instance of a Proposal that is active on a node.
In the OpenStack Identity context: concept of controlling the actions or set of operations that a user is allowed to perform. A role includes a set of rights and privileges. A user assuming that role inherits those rights and privileges.
An object storage by Amazon that can be used to store and retrieve data on the Web. Similar in function to OpenStack Object Storage. It can act as a back-end store for Glance images.
A service model of cloud computing where applications are hosted by a service provider and made available to customers remotely as a Web-based service.
In an environment where all nodes of a High Availability cluster have access to shared storage, a small partition is used for disk-based fencing.
Concept in OpenStack Networking. A security group is a container for security group rules. Security group rules allow to specify the type of traffic and direction (ingress/egress) that is allowed to pass through a port.
An individual piece of equipment or software which will cause system downtime or data loss if it fails. To eliminate single points of failure, High Availability systems seek to provide redundancy for crucial pieces of equipment or software.
When you first boot a node in SUSE OpenStack Cloud Crowbar via PXE, it is booted with the SLEShammer image. This performs the initial hardware discovery, and registers the node with Crowbar. After you allocate the node, it is rebooted with a regular SLES installation image.
See Volume Snapshot or Instance Snapshot.
Also known as a “partitioned cluster” scenario. Either through a software or hardware failure, the cluster nodes are divided into two or more groups that do not know of each other. STONITH prevents a split brain situation from badly affecting the entire cluster.
A service where subsequent requests to the service depend on the results of the first request.
A service that provides a response after your request, and then requires no further attention.
The acronym for “Shoot the other node in the head”. It refers to the fencing mechanism that shuts down a misbehaving node to prevent it from causing trouble in a cluster.
Node within a SUSE OpenStack Cloud. Acts as the controller for cloud-based storage. A cloud can contain multiple Storage Nodes.
In the OpenStack Networking API: A block of IP addresses and other network configuration (for example, a default gateway, DNS servers) that can be associated with an OpenStack Networking network. Each subnet represents an IPv4 or IPv6 address block. Multiple subnets can be associated with a network, if necessary.
An integrated suite of open source clustering technologies that enables you to implement highly available physical and virtual Linux clusters.
User role in SUSE OpenStack Cloud Crowbar. Manages projects, users, images, flavors, and quotas within SUSE OpenStack Cloud Crowbar.
The SUSE® OpenStack Cloud Crowbar Dashboard is a Web interface that enables cloud
administrators and users to manage various OpenStack services. It is based
on OpenStack Dashboard (also known under its codename
Horizon
).
User role in SUSE OpenStack Cloud Crowbar. Installs and deploys SUSE OpenStack Cloud Crowbar.
User role in SUSE OpenStack Cloud Crowbar. End user who launches and manages instances, can create snapshots, and use volumes for persistent storage within SUSE OpenStack Cloud Crowbar.
Code name for OpenStack Object Storage.
A virtual networking device. A TAP device, such as
vnet0
is how hypervisors such as KVM and
Xen implement a virtual network interface card (vNIC). An Ethernet
frame sent to a TAP device is received by the guest operating system.
The tap option connects the network stack of the guest operating system
to a TAP network device on the host.
A module (code name: Ceilometer
) for metering
OpenStack-based clouds. The project aims to provide a unique point of
contact across all OpenStack core components for acquiring metrics.
The metrics can then be consumed by other components such as customer billing.
The module is an integrated project of OpenStack as of the Havana
release.
See Project.
A technology for sending messages to a single network destination. Corosync supports both multicast and unicast. In Corosync, unicast is implemented as UDP-unicast (UDPU).
In the OpenStack context, a digital representation of a person, system, or service who uses OpenStack cloud services. Users can be directly assigned to a particular project and behave as if they are contained in that project.
A virtual networking device. The acronym veth stands for virtual Ethernet interface. A veth is a pair of virtual network interfaces correctly directly together. An Ethernet frame sent to one end of a veth pair is received by the other end of a veth pair. OpenStack Networking uses veth pairs as virtual patch cables to make connections between virtual bridges.
A physical method for network virtualization. VLANs allow to create virtual networks across a distributed network. Disparate hosts (on independent networks) appear as if they were part of the same broadcast domain.
An operating system instance that runs on top of a hypervisor. Multiple virtual machines can run on the same physical host at the same time.
Virtual network interface card.
Detachable block storage device. Unlike a SAN, it can only be attached to one instance at a time.
A virtual disk consisting of aggregated physical volumes. Volume groups can be logically partitioned into logical volumes.
A point-in-time copy of an OpenStack storage volume. Used to back up volumes.
A software that runs on a host or node and provides the features and functions of a hardware-based network switch.
A logical grouping of Compute services and virtual machine hosts.