9 Ceph Object Gateway #
Ceph Object Gateway is an object storage interface built on top of
librgw
to provide applications with a RESTful gateway to
Ceph clusters. It supports two interfaces:
S3-compatible: Provides object storage functionality with an interface that is compatible with a large subset of the Amazon S3 RESTful API.
Swift-compatible: Provides object storage functionality with an interface that is compatible with a large subset of the OpenStack Swift API.
The Object Gateway daemon uses an embedded HTTP server (CivetWeb) for interacting with the Ceph cluster. Since it provides interfaces compatible with OpenStack Swift and Amazon S3, the Object Gateway has its own user management. Object Gateway can store data in the same cluster that is used to store data from CephFS clients or RADOS Block Device clients. The S3 and Swift APIs share a common name space, so you may write data with one API and retrieve it with the other.
Important: Object Gateway Deployed by DeepSea
Since SUSE Enterprise Storage 5.5, the Object Gateway is installed as a DeepSea role, therefore you do not need to install it manually.
To install the Object Gateway during the cluster deployment, see Section 4.3, “Cluster Deployment”.
To add a new node with Object Gateway to the cluster, see Section 1.2, “Adding New Roles to Nodes”.
9.1 Object Gateway Manual Installation #
Install Object Gateway on a node that is not using port 80. For example a node already running openATTIC is already using port 80. The following command installs all required components:
root #
zypper ref && zypper in ceph-radosgwIf the Apache server from the previous Object Gateway instance is running, stop it and disable the relevant service:
root #
systemctl stop disable apache2.serviceEdit
/etc/ceph/ceph.conf
and add the following lines:[client.rgw.gateway_host] rgw frontends = "civetweb port=80"
Tip
If you want to configure Object Gateway/CivetWeb for use with SSL encryption, modify the line accordingly:
rgw frontends = civetweb port=7480s ssl_certificate=path_to_certificate.pem
Restart the Object Gateway service.
root #
systemctl restart ceph-radosgw@rgw.gateway_host
9.1.1 Object Gateway Configuration #
Several steps are required to configure an Object Gateway.
9.1.1.1 Basic Configuration #
Configuring a Ceph Object Gateway requires a running Ceph Storage Cluster. The Ceph Object Gateway is a client of the Ceph Storage Cluster. As a Ceph Storage Cluster client, it requires:
A host name for the gateway instance, for example
gateway
.A storage cluster user name with appropriate permissions and a keyring.
Pools to store its data.
A data directory for the gateway instance.
An instance entry in the Ceph configuration file.
Each instance must have a user name and key to communicate with a Ceph storage cluster. In the following steps, we use a monitor node to create a bootstrap keyring, then create the Object Gateway instance user keyring based on the bootstrap one. Then, we create a client user name and key. Next, we add the key to the Ceph Storage Cluster. Finally, we distribute the keyring to the node containing the gateway instance.
Create a keyring for the gateway:
root #
ceph-authtool --create-keyring /etc/ceph/ceph.client.rgw.keyringroot #
chmod +r /etc/ceph/ceph.client.rgw.keyringGenerate a Ceph Object Gateway user name and key for each instance. As an example, we will use the name
gateway
afterclient.radosgw
:root #
ceph-authtool /etc/ceph/ceph.client.rgw.keyring \ -n client.rgw.gateway --gen-keyAdd capabilities to the key:
root #
ceph-authtool -n client.rgw.gateway --cap osd 'allow rwx' \ --cap mon 'allow rwx' /etc/ceph/ceph.client.rgw.keyringOnce you have created a keyring and key to enable the Ceph Object Gateway with access to the Ceph Storage Cluster, add the key to your Ceph Storage Cluster. For example:
root #
ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.gateway \ -i /etc/ceph/ceph.client.rgw.keyringDistribute the keyring to the node with the gateway instance:
root #
scp /etc/ceph/ceph.client.rgw.keyring ceph@hostname:/home/cephcephadm >
ssh hostnameroot #
mv ceph.client.rgw.keyring /etc/ceph/ceph.client.rgw.keyring
Tip: Use Bootstrap Keyring
An alternative way is to create the Object Gateway bootstrap keyring, and then create the Object Gateway keyring from it:
Create an Object Gateway bootstrap keyring on one of the monitor nodes:
root #
ceph \ auth get-or-create client.bootstrap-rgw mon 'allow profile bootstrap-rgw' \ --connect-timeout=25 \ --cluster=ceph \ --name mon. \ --keyring=/var/lib/ceph/mon/ceph-node_host/keyring \ -o /var/lib/ceph/bootstrap-rgw/keyringCreate the
/var/lib/ceph/radosgw/ceph-rgw_name
directory for storing the bootstrap keyring:root #
mkdir \ /var/lib/ceph/radosgw/ceph-rgw_nameCreate an Object Gateway keyring from the newly created bootstrap keyring:
root #
ceph \ auth get-or-create client.rgw.rgw_name osd 'allow rwx' mon 'allow rw' \ --connect-timeout=25 \ --cluster=ceph \ --name client.bootstrap-rgw \ --keyring=/var/lib/ceph/bootstrap-rgw/keyring \ -o /var/lib/ceph/radosgw/ceph-rgw_name/keyringCopy the Object Gateway keyring to the Object Gateway host:
root #
scp \ /var/lib/ceph/radosgw/ceph-rgw_name/keyring \ rgw_host:/var/lib/ceph/radosgw/ceph-rgw_name/keyring
9.1.1.2 Create Pools (Optional) #
Ceph Object Gateways require Ceph Storage Cluster pools to store specific gateway data. If the user you created has proper permissions, the gateway will create the pools automatically. However, ensure that you have set an appropriate default number of placement groups per pool in the Ceph configuration file.
The pool names follow the
ZONE_NAME.POOL_NAME
syntax. When configuring a gateway with the default region and zone, the
default zone name is 'default' as in our example:
.rgw.root default.rgw.control default.rgw.meta default.rgw.log default.rgw.buckets.index default.rgw.buckets.data
To create the pools manually, see Section 8.2.2, “Create a Pool”.
Important: Object Gateway and Erasure-coded Pools
Only the default.rgw.buckets.data
pool can be
erasure-coded. All other pools need to be replicated, otherwise the
gateway is not accessible.
9.1.1.3 Adding Gateway Configuration to Ceph #
Add the Ceph Object Gateway configuration to the Ceph Configuration file. The Ceph Object Gateway configuration requires you to identify the Ceph Object Gateway instance. Then, specify the host name where you installed the Ceph Object Gateway daemon, a keyring (for use with cephx), and optionally a log file. For example:
[client.rgw.instance-name] host = hostname keyring = /etc/ceph/ceph.client.rgw.keyring
Tip: Object Gateway Log File
To override the default Object Gateway log file, include the following:
log file = /var/log/radosgw/client.rgw.instance-name.log
The [client.rgw.*]
portion of the gateway instance
identifies this portion of the Ceph configuration file as configuring a
Ceph Storage Cluster client where the client type is a Ceph Object Gateway
(radosgw). The instance name follows. For example:
[client.rgw.gateway] host = ceph-gateway keyring = /etc/ceph/ceph.client.rgw.keyring
Note
The host must be your machine host name, excluding the domain name.
Then turn off print continue
. If you have it set to
true, you may encounter problems with PUT operations:
rgw print continue = false
To use a Ceph Object Gateway with subdomain S3 calls (for example
http://bucketname.hostname
), you must add the Ceph
Object Gateway DNS name under the [client.rgw.gateway]
section
of the Ceph configuration file:
[client.rgw.gateway] ... rgw dns name = hostname
You should also consider installing a DNS server such as Dnsmasq on your
client machine(s) when using the
http://bucketname.hostname
syntax. The dnsmasq.conf
file should include the
following settings:
address=/hostname/host-ip-address listen-address=client-loopback-ip
Then, add the client-loopback-ip IP address as the first DNS server on the client machine(s).
9.1.1.4 Create Data Directory #
Deployment scripts may not create the default Ceph Object Gateway data directory.
Create data directories for each instance of a radosgw daemon if not
already done. The host
variables in the Ceph
configuration file determine which host runs each instance of a radosgw
daemon. The typical form specifies the radosgw daemon, the cluster name,
and the daemon ID.
root #
mkdir -p /var/lib/ceph/radosgw/cluster-id
Using the example ceph.conf
settings above, you would
execute the following:
root #
mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway
9.1.1.5 Restart Services and Start the Gateway #
To ensure that all components have reloaded their configurations, we
recommend restarting your Ceph Storage Cluster service. Then, start up
the radosgw
service. For more information, see
Chapter 2, Introduction and
Section 13.3, “Operating the Object Gateway Service”.
When the service is up and running, you can make an anonymous GET request to see if the gateway returns a response. A simple HTTP request to the domain name should return the following:
<ListAllMyBucketsResult> <Owner> <ID>anonymous</ID> <DisplayName/> </Owner> <Buckets/> </ListAllMyBucketsResult>