Ceph Object Gateway | Administration Guide | SUSE Enterprise Storage 5.5 (SES 5 & SES 5.5)

Applies to SUSE Enterprise Storage 5.5 (SES 5 & SES 5.5)

13 Ceph Object Gateway #

This chapter introduces details about administration tasks related to Object Gateway, such as checking status of the service, managing accounts, multisite gateways, or LDAP authentication.

13.1 Object Gateway Restrictions and Naming Limitations #

Following is a list of important Object Gateway limits:

13.1.1 Bucket Limitations #

When approaching Object Gateway via the S3 API, bucket names are limited to DNS-compliant names with a dash character '-' allowed. When approaching Object Gateway via the Swift API, you may use any combination of UTF-8 supported characters except for a slash character '/'. The maximum length of a bucket name is 255 characters. Bucket names must be unique.

Tip: Use DNS-compliant Bucket Names

Although you may use any UTF-8 based bucket name via the Swift API, it is recommended to name buckets with regard to the S3 naming limitations to avoid problems accessing the same bucket via the S3 API.

13.1.2 Stored Object Limitations #

Maximum number of object per user: No restriction by default (limited by ~ 2^63).
Maximum number of object per bucket: No restriction by default (limited by ~ 2^63).
Maximum size of an object to upload / store: Single uploads are restricted to 5GB. Use multipart for larger object sizes. The maximum number of multipart chunks is 10000.

13.1.3 HTTP Header Limitations #

HTTP header and request limitation depend on the Web front-end used. The default CivetWeb restricts the number of HTTP headers to 64 headers, and the size of the HTTP header to 16kB.

13.2 Deploying the Object Gateway #

The recommended way of deploying the Ceph Object Gateway is via the DeepSea infrastructure by adding the relevant role-rgw [...] line(s) into the policy.cfg file on the Salt master, and running required DeepSea stages.

To include the Object Gateway during the Ceph cluster deployment process, refer to Section 4.3, “Cluster Deployment” and Section 4.5.1, “The policy.cfg File”.
To add the Object Gateway role to an already deployed cluster, refer to Section 1.2, “Adding New Roles to Nodes”.

13.3 Operating the Object Gateway Service #

Object Gateway service is operated with the systemctl command. You need to have root privileges to operate the Object Gateway service. Note that gateway_host is the host name of the server whose Object Gateway instance you need to operate.

The following subcommands are supported for the Object Gateway service:

systemctl status ceph-radosgw@rgw.gateway_host: Prints the status information of the service.
systemctl start ceph-radosgw@rgw.gateway_host: Starts the service if it is not already running.
systemctl restart ceph-radosgw@rgw.gateway_host: Restarts the service.
systemctl stop ceph-radosgw@rgw.gateway_host: Stops the running service.
systemctl enable ceph-radosgw@rgw.gateway_host: Enables the service so that it is automatically started on system start-up.
systemctl disable ceph-radosgw@rgw.gateway_host: Disables the service so that it is not automatically started on system start-up.

13.4 Configuration Parameters #

You can influence the Object Gateway behavior by a number of options in the ceph.conf file under the section named

[client.radosgw.INSTANCE_NAME]

If an option is not specified, its default value is used. A complete list of the Object Gateway options follows:

General Settings #

rgw frontends

Configures the HTTP front end(s). Specify multiple front ends in a comma-delimited list. Each front end configuration may include a list of options separated by spaces, where each option is in the form “key=value” or “key”. Default is

rgw frontends = civetweb port=7480

Note: `tcp_nodelay`

This option may affect the transfer rate of sending TCP packets, depending on the data chunk sizes. If set to '1', the socket option will disable Nagle's algorithm on the connection. Therefore packets will be sent as soon as possible instead of waiting for a full buffer or timeout to occur.

rgw data

Sets the location of the data files for the Object Gateway. Default is /var/lib/ceph/radosgw/CLUSTER_ID.

rgw enable apis

Enables the specified APIs. Default is 's3, swift, swift_auth, admin All APIs'.

rgw cache enabled

Enables or disables the Object Gateway cache. Default is 'true'.

rgw cache lru size

The number of entries in the Object Gateway cache. Default is 10000.

rgw socket path

The socket path for the domain socket. FastCgiExternalServer uses this socket. If you do not specify a socket path, the Object Gateway will not run as an external server. The path you specify here needs to be the same as the path specified in the rgw.conf file.

rgw fcgi socket backlog

The socket backlog for fcgi. Default is 1024.

rgw host

The host for the Object Gateway instance. It can be an IP address or a hostname. Default is 0.0.0.0

rgw port

The port number where the instance listens for requests. If not specified, the Object Gateway runs external FastCGI.

rgw dns name

The DNS name of the served domain.

rgw script uri

The alternative value for the SCRIPT_URI if not set in the request.

rgw request uri

The alternative value for the REQUEST_URI if not set in the request.

rgw print continue

Enable 100-continue if it is operational. Default is 'true'.

rgw remote addr param

The remote address parameter. For example, the HTTP field containing the remote address, or the X-Forwarded-For address if a reverse proxy is operational. Default is REMOTE_ADDR.

rgw op thread timeout

The timeout in seconds for open threads. Default is 600.

rgw op thread suicide timeout

The time timeout in seconds before the Object Gateway process dies. Disabled if set to 0 (default).

rgw thread pool size

Number of threads for the CivetWeb server. Increase to a higher value if you need to serve more requests. Defaults to 100 threads.

rgw num rados handles

The number of RADOS cluster handles for Object Gateway. Having a configurable number of RADOS handles results in significant performance boost for all types of workloads. Each Object Gateway worker thread now gets to pick a RADOS handle for its lifetime. Default is 1.

rgw num control oids

The number of notification objects used for cache synchronization between different rgw instances. Default is 8.

rgw init timeout

The number of seconds before the Object Gateway gives up on initialization. Default is 30.

rgw mime types file

The path and location of the MIME types. Used for Swift auto-detection of object types. Default is /etc/mime.types.

rgw gc max objs

The maximum number of objects that may be handled by garbage collection in one garbage collection processing cycle. Default is 32.

rgw gc obj min wait

The minimum wait time before the object may be removed and handled by garbage collection processing. Default is 2 * 3600.

rgw gc processor max time

The maximum time between the beginning of two consecutive garbage collection processing cycles. Default is 3600.

rgw gc processor period

The cycle time for garbage collection processing. Default is 3600.

rgw s3 success create obj status

The alternate success status response for create-obj. Default is 0.

rgw resolve cname

Whether the Object Gateway should use DNS CNAME record of the request host name field (if host name is not equal to the Object Gateway DNS name). Default is 'false'.

rgw obj stripe size

The size of an object stripe for Object Gateway objects. Default is 4 << 20.

rgw extended http attrs

Add a new set of attributes that can be set on an entity (for example a user, a bucket or an object). These extra attributes can be set through HTTP header fields when putting the entity or modifying it using POST method. If set, these attributes will return as HTTP fields when requesting GET/HEAD on the entity. Default is 'content_foo, content_bar, x-foo-bar'.

rgw exit timeout secs

Number of seconds to wait for a process before exiting unconditionally. Default is 120.

rgw get obj window size

The window size in bytes for a single object request. Default is '16 << 20'.

rgw get obj max req size

The maximum request size of a single GET operation sent to the Ceph Storage Cluster. Default is 4 << 20.

rgw relaxed s3 bucket names

Enables relaxed S3 bucket names rules for US region buckets. Default is 'false'.

rgw list buckets max chunk

The maximum number of buckets to retrieve in a single operation when listing user buckets. Default is 1000.

rgw override bucket index max shards

Represents the number of shards for the bucket index object. Setting 0 (default) indicates there is no sharding. It is not recommended to set a value too large (for example 1000) as it increases the cost for bucket listing. This variable should be set in the client or global sections so that it is automatically applied to radosgw-admin commands.

rgw curl wait timeout ms

The timeout in milliseconds for certain curl calls. Default is 1000.

rgw copy obj progress

Enables output of object progress during long copy operations. Default is 'true'.

rgw copy obj progress every bytes

The minimum bytes between copy progress output. Default is 1024 * 1024.

rgw admin entry

The entry point for an admin request URL. Default is 'admin'.

rgw content length compat

Enable compatibility handling of FCGI requests with both CONTENT_LENGTH AND HTTP_CONTENT_LENGTH set. Default is 'false'.

rgw bucket quota ttl

The amount of time in seconds that cached quota information is trusted. After this timeout, the quota information will be re-fetched from the cluster. Default is 600.

rgw user quota bucket sync interval

The amount of time in seconds for which the bucket quota information is accumulated before syncing to the cluster. During this time, other Object Gateway instances will not see the changes in the bucket quota stats related to operations on this instance. Default is 180.

rgw user quota sync interval

The amount of time in seconds for which user quota information is accumulated before syncing to the cluster. During this time, other Object Gateway instances will not see the changes in the user quota stats related to operations on this instance. Default is 180.

rgw bucket default quota max objects

Default maximum number of objects per bucket. It is set on new users if no other quota is specified, and has no effect on existing users. This variable should be set in the client or global sections so that it is automatically applied to radosgw-admin commands. Default is -1.

rgw bucket default quota max size

Default maximum capacity per bucket in bytes. It is set on new users if no other quota is specified, and has no effect on existing users. Default is -1.

rgw user default quota max objects

Default maximum number of objects for a user. This includes all objects in all buckets owned by the user. It is set on new users if no other quota is specified, and has no effect on existing users. Default is -1.

rgw user default quota max size

The value for user maximum size quota in bytes set on new users if no other quota is specified. It has no effect on existing users. Default is -1.

rgw verify ssl

Verify SSL certificates while making requests. Default is 'true'.

rgw max chunk size

Maximum size of a chunk of data that will be read in a single operation. Increasing the value to 4MB (4194304) will provide better performance when processing large objects. Default is 128kB (131072).

Multisite Settings #

rgw zone: The name of the zone for the gateway instance. If no zone is set, a cluster-wide default can be configured with the radosgw-admin zone default command.
rgw zonegroup: The name of the zonegroup for the gateway instance. If no zonegroup is set, a cluster-wide default can be configured with the radosgw-admin zonegroup default command.
rgw realm: The name of the realm for the gateway instance. If no realm is set, a cluster-wide default can be configured with theradosgw-admin realm default command.
rgw run sync thread: If there are other zones in the realm to synchronize from, spawn threads to handle the synchronization of data and metadata. Default is 'true'.
rgw data log window: The data log entries window in seconds. Default is 30/
rgw data log changes size: The number of in-memory entries to hold for the data changes log. Default is 1000.
rgw data log obj prefix: The object name prefix for the data log. Default is 'data_log'.
rgw data log num shards: The number of shards (objects) on which to keep the data changes log. Default is 128.
rgw md log max shards: The maximum number of shards for the metadata log. Default is 64.

Swift Settings #

rgw enforce swift acls: Enforces the Swift Access Control List (ACL) settings. Default is 'true'.
rgw swift token expiration: The time in seconds for expiring a Swift token. Default is 24 * 3600.
rgw swift url: The URL for the Ceph Object Gateway Swift API.
rgw swift url prefix: The URL prefix for the Swift StorageURL that goes in front of the “/v1” part. This allows to run several Gateway instances on the same host. For compatibility, setting this configuration variable to empty causes the default “/swift” to be used. Use explicit prefix “/” to start StorageURL at the root.
Warning
Setting this option to “/” will not work if S3 API is enabled. Keep in mind that disabling S3 will make impossible to deploy the Object Gateway in the multisite configuration!
rgw swift auth url: Default URL for verifying v1 authentication tokens when the internal Swift authentication is not used.
rgw swift auth entry: The entry point for a Swift authentication URL. Default is 'auth'.
rgw swift versioning enabled: Enables the Object Versioning of OpenStack Object Storage API. This allows clients to put the X-Versions-Location attribute on containers that should be versioned. The attribute specifies the name of container storing archived versions. It must be owned by the same user that the versioned container due to access control verification - ACLs are not taken into consideration. Those containers cannot be versioned by the S3 object versioning mechanism. Default is 'false'.

Logging Settings #

rgw log nonexistent bucket: Enables the Object Gateway to log a request for a non-existent bucket. Default is 'false'.
rgw log object name: The logging format for an object name. See the manual page man 1 date for details about format specifiers. Default is '%Y-%m-%d-%H-%i-%n'.
rgw log object name utc: Whether a logged object name includes a UTC time. If set to 'false' (default), it uses the local time.
rgw usage max shards: The maximum number of shards for usage logging. Default is 32.
rgw usage max user shards: The maximum number of shards used for a single user’s usage logging. Default is 1.
rgw enable ops log: Enable logging for each successful Object Gateway operation. Default is 'false'.
rgw enable usage log: Enable the usage log. Default is 'false'.
rgw ops log rados: Whether the operations log should be written to the Ceph Storage Cluster back end. Default is 'true'.
rgw ops log socket path: The Unix domain socket for writing operations logs.
rgw ops log data backlog: The maximum data backlog data size for operations logs written to a Unix domain socket. Default is 5 << 20.
rgw usage log flush threshold: The number of dirty merged entries in the usage log before flushing synchronously. Default is 1024.
rgw usage log tick interval: Flush pending usage log data every 'n' seconds. Default is 30.
rgw log http headers: Comma-delimited list of HTTP headers to include in log entries. Header names are case insensitive, and use the full header name with words separated by underscores. For example 'http_x_forwarded_for, http_x_special_k'.
rgw intent log object name: The logging format for the intent log object name. See the manual page man 1 date for details about format specifiers. Default is '%Y-%m-%d-%i-%n'.
rgw intent log object name utc: Whether the intent log object name includes a UTC time. If set to 'false' (default), it uses the local time.

Keystone Settings #

rgw keystone url: The URL for the Keystone server.
rgw keystone api version: The version (2 or 3) of OpenStack Identity API that should be used for communication with the Keystone server. Default is 2.
rgw keystone admin domain: The name of the OpenStack domain with the administrator privilege when using OpenStack Identity API v3.
rgw keystone admin project: The name of the OpenStack project with the administrator privilege when using OpenStack Identity API v3. If not set, the value of the rgw keystone admin tenant will be used instead.
rgw keystone admin token: The Keystone administrator token (shared secret). In the Object Gateway, authentication with the administrator token has priority over authentication with the administrator credentials (options rgw keystone admin user, rgw keystone admin password, rgw keystone admin tenant, rgw keystone admin project, and rgw keystone admin domain). Administrator token feature is considered as deprecated.
rgw keystone admin tenant: The name of the OpenStack tenant with the administrator privilege (Service Tenant) when using OpenStack Identity API v2.
rgw keystone admin user: The name of the OpenStack user with the administrator privilege for Keystone authentication (Service User) when using OpenStack Identity API v2.
rgw keystone admin password: The password for the OpenStack administrator user when using OpenStack Identity API v2.
rgw keystone accepted roles: The roles required to serve requests. Default is 'Member, admin'.
rgw keystone token cache size: The maximum number of entries in each Keystone token cache. Default is 10000.
rgw keystone revocation interval: The number of seconds between token revocation checks. Default is 15 * 60.
rgw keystone verify ssl: Verify SSL certificates while making token requests to keystone. Default is 'true'.

13.4.1 Additional Notes #

rgw dns name: If the parameter rgw dns name is added to the ceph.conf, make sure that the S3 client is configured to direct requests at the endpoint specified by rgw dns name.

13.5 Managing Object Gateway Access #

You can communicate with Object Gateway using either S3- or Swift-compatible interface. S3 interface is compatible with a large subset of the Amazon S3 RESTful API. Swift interface is compatible with a large subset of the OpenStack Swift API.

Both interfaces require you to create a specific user, and install the relevant client software to communicate with the gateway using the user's secret key.

13.5.1 Accessing Object Gateway #

13.5.1.1 S3 Interface Access #

To access the S3 interface, you need a REST client. S3cmd is a command line S3 client. You can find it in the OpenSUSE Build Service. The repository contains versions for both SUSE Linux Enterprise and openSUSE based distributions.

If you want to test your access to the S3 interface, you can also write a small a Python script. The script will connect to Object Gateway, create a new bucket, and list all buckets. The values for aws_access_key_id and aws_secret_access_key are taken from the values of access_key and secret_key returned by the radosgw_admin command from Section 13.5.2.1, “Adding S3 and Swift Users”.

Install the python-boto package:
```
root # zypper in python-boto
```

Create a new Python script called s3test.py with the following content:

import boto
import boto.s3.connection
access_key = '11BS02LGFB6AL6H1ADMW'
secret_key = 'vzCEkuryfn060dfee4fgQPqFrncKEIkh3ZcdOANY'
conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = '{hostname}',
is_secure=False,
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)
bucket = conn.create_bucket('my-new-bucket')
for bucket in conn.get_all_buckets():
  print "{name}\t{created}".format(
  name = bucket.name,
  created = bucket.creation_date,
  )

Replace {hostname} with the host name of the host where you configured Object Gateway service, for example gateway_host.

Run the script:
```
python s3test.py
```
The script outputs something like the following:
```
my-new-bucket 2015-07-22T15:37:42.000Z
```

13.5.1.2 Swift Interface Access #

To access Object Gateway via Swift interface, you need the swift command line client. Its manual page man 1 swift tells you more about its command line options.

The package is included in the 'Public Cloud' module for SUSE Linux Enterprise 12 SP3. Before installing the package, you need to activate the module and refresh the software repository:

root # SUSEConnect -p sle-module-public-cloud/12/x86_64
sudo zypper refresh

To install the swift command, run the following:

root # zypper in python-swiftclient

The swift access uses the following syntax:

cephadm > swift -A http://IP_ADDRESS/auth/1.0 \
-U example_user:swift -K 'swift_secret_key' list

Replace IP_ADDRESS with the IP address of the gateway server, and swift_secret_key with its value from the output of the radosgw-admin key create command executed for the swift user in Section 13.5.2.1, “Adding S3 and Swift Users”.

For example:

cephadm > swift -A http://gateway.example.com/auth/1.0 -U example_user:swift \
-K 'r5wWIxjOCeEO7DixD1FjTLmNYIViaC6JVhi3013h' list

The output is:

my-new-bucket

13.5.2 Managing S3 and Swift Accounts #

13.5.2.1 Adding S3 and Swift Users #

You need to create a user, access key and secret to enable end users to interact with the gateway. There are two types of users: a user and subuser. While users are used when interacting with the S3 interface, subusers are users of the Swift interface. Each subuser is associated to a user.

Users can also be added via the DeepSea file rgw.sls. For an example, see Section 16.3.1, “Different Object Gateway Users for NFS Ganesha”.

To create a Swift user, follow the steps:

To create a Swift user—which is a subuser in our terminology—you need to create the associated user first.

cephadm > radosgw-admin user create --uid=username \
 --display-name="display-name" --email=email

For example:

cephadm > radosgw-admin user create \
   --uid=example_user \
   --display-name="Example User" \
   --email=penguin@example.com

To create a subuser (Swift interface) for the user, you must specify the user ID (--uid=username), a subuser ID, and the access level for the subuser.

cephadm > radosgw-admin subuser create --uid=uid \
 --subuser=uid \
 --access=[ read | write | readwrite | full ]

For example:

cephadm > radosgw-admin subuser create --uid=example_user \
 --subuser=example_user:swift --access=full

Generate a secret key for the user.

cephadm > radosgw-admin key create \
   --gen-secret \
   --subuser=example_user:swift \
   --key-type=swift

Both commands will output JSON-formatted data showing the user state. Notice the following lines, and remember the secret_key value:
```
"swift_keys": [
   { "user": "example_user:swift",
     "secret_key": "r5wWIxjOCeEO7DixD1FjTLmNYIViaC6JVhi3013h"}],
```

When accessing Object Gateway through the S3 interface you need to create a S3 user by running:

cephadm > radosgw-admin user create --uid=username \
 --display-name="display-name" --email=email

For example:

cephadm > radosgw-admin user create \
   --uid=example_user \
   --display-name="Example User" \
   --email=penguin@example.com

The command also creates the user's access and secret key. Check its output for access_key and secret_key keywords and their values:

[...]
 "keys": [
       { "user": "example_user",
         "access_key": "11BS02LGFB6AL6H1ADMW",
         "secret_key": "vzCEkuryfn060dfee4fgQPqFrncKEIkh3ZcdOANY"}],
 [...]

13.5.2.2 Removing S3 and Swift Users #

The procedure for deleting users is similar for S3 and Swift users. But in case of Swift users you may need to delete the user including its subusers.

To remove a S3 or Swift user (including all its subusers), specify user rm and the user ID in the following command:

cephadm > radosgw-admin user rm --uid=example_user

To remove a subuser, specify subuser rm and the subuser ID.

cephadm > radosgw-admin subuser rm --uid=example_user:swift

You can make use of the following options:

--purge-data: Purges all data associated to the user ID.
--purge-keys: Purges all keys associated to the user ID.

Tip: Removing a Subuser

When you remove a subuser, you are removing access to the Swift interface. The user will remain in the system.

13.5.2.3 Changing S3 and Swift User Access and Secret Keys #

The access_key and secret_key parameters identify the Object Gateway user when accessing the gateway. Changing the existing user keys is the same as creating new ones, as the old keys get overwritten.

For S3 users, run the following:

cephadm > radosgw-admin key create --uid=example_user --key-type=s3 --gen-access-key --gen-secret

For Swift users, run the following:

cephadm > radosgw-admin key create --subuser=example_user:swift --key-type=swift --gen-secret

--key-type=type: Specifies the type of key. Either swift or s3.
--gen-access-key: Generates a random access key (for S3 user by default).
--gen-secret: Generates a random secret key.
--secret=key: Specifies a secret key, for example manually generated.

13.5.2.4 User Quota Management #

The Ceph Object Gateway enables you to set quotas on users and buckets owned by users. Quotas include the maximum number of objects in a bucket and the maximum storage size in megabytes.

Before you enable a user quota, you first need to set its parameters:

cephadm > radosgw-admin quota set --quota-scope=user --uid=example_user \
 --max-objects=1024 --max-size=1024

--max-objects: Specifies the maximum number of objects. A negative value disables the check.
--max-size: Specifies the maximum number of bytes. A negative value disables the check.
--quota-scope: Sets the scope for the quota. The options are bucket and user. Bucket quotas apply to buckets a user owns. User quotas apply to a user.

Once you set a user quota, you may enable it:

cephadm > radosgw-admin quota enable --quota-scope=user --uid=example_user

To disable a quota:

cephadm > radosgw-admin quota disable --quota-scope=user --uid=example_user

To list quota settings:

cephadm > radosgw-admin user info --uid=example_user

To update quota statistics:

cephadm > radosgw-admin user stats --uid=example_user --sync-stats

13.6 Enabling HTTPS/SSL for Object Gateways #

To enable the default Object Gateway role to communicate securely using SSL, you need to either have a CA-issued certificate, or create a self-signed one— not both. There are two ways to configure Object Gateway with HTTPS enabled: a simple way that makes use of the default settings, and an advanced way that lets you fine-tune HTTPS related settings.

13.6.1 Create a Self-Signed Certificate #

By default, DeepSea expects the certificate file in /srv/salt/ceph/rgw/cert/rgw.pem on the Salt master. It will then distribute the certificate to /etc/ceph/rgw.pem on the Salt minion with the Object Gateway role, where Ceph reads it.

The following procedure describes how to generate a self-signed SSL certificate on the Salt master node.

Note

If you have a valid certificated signed by a CA, proceed to Step 3.

If you need your Object Gateway to be known by additional subject identities, add them to the subjectAltName option in the [v3_req] section of the /etc/ssl/openssl.cnf file:
```
[...]
[ v3_req ]
subjectAltName = DNS:server1.example.com DNS:server2.example.com
[...]
```
Tip: IP Addresses in subjectAltName
To use IP addresses instead of domain names in the subjectAltName option, replace the example line with the following:
```
subjectAltName = IP:10.0.0.10 IP:10.0.0.11
```
Create the key and the certificate using openssl. Enter all data you need to include in your certificate. We recommend entering the FQDN as the common name. Before signing the certificate, verify that 'X509v3 Subject Alternative Name:' is included in requested extensions, and that the resulting certificate has "X509v3 Subject Alternative Name:" set.
```
root@master # openssl req -x509 -nodes -days 1095 \
 -newkey rsa:4096 -keyout rgw.key -out /srv/salt/ceph/rgw/cert/rgw.pem
```

Append the files to the rgw.pem. For example:

root@master # cat rgw.key >> /srv/salt/ceph/rgw/cert/rgw.pem

13.6.2 Simple HTTPS Configuration #

By default, Ceph on the Object Gateway node reads the /etc/ceph/rgw.pem certificate, and uses port 443 for secure SSL communication. If you do not need to change these values, follow these steps:

Edit /srv/pillar/ceph/stack/global.yml and add the following line:
```
rgw_init: default-ssl
```

Copy the default Object Gateway SSL configuration to the ceph.conf.d subdirectory:

root@master # cp /srv/salt/ceph/configuration/files/rgw-ssl.conf \
 /srv/salt/ceph/configuration/files/ceph.conf.d/rgw.conf

Run DeepSea Stages 2, 3, and 4 to apply the changes:

root@master # salt-run state.orch ceph.stage.2
root@master # salt-run state.orch ceph.stage.3
root@master # salt-run state.orch ceph.stage.4

13.6.3 Advanced HTTPS Configuration #

If you need to change the default values for SSL settings of the Object Gateway, follow these steps:

Edit /srv/pillar/ceph/stack/global.yml and add the following line:
```
rgw_init: default-ssl
```

Copy the default Object Gateway SSL configuration to the ceph.conf.d subdirectory:

root@master # cp /srv/salt/ceph/configuration/files/rgw-ssl.conf \
 /srv/salt/ceph/configuration/files/ceph.conf.d/rgw.conf

Edit /srv/salt/ceph/configuration/files/ceph.conf.d/rgw.conf and change the default options, such as port number or path to the SSL certificate, to reflect your setup.

Run DeepSea Stage 3 and 4 to apply the changes:

root@master # salt-run state.orch ceph.stage.3
root@master # salt-run state.orch ceph.stage.4

Tip: Binding to Multiple Ports

The CivetWeb server can bind to multiple ports. This is useful if you need to access a single Object Gateway instance with both SSL and non-SSL connections. When specifying the ports, separate their numbers by a plus sign '+'. A two-port configuration line example follows:

[client.{{ client }}]
rgw_frontends = civetweb port=80+443s ssl_certificate=/etc/ceph/rgw.pem

13.7 Sync Modules #

The multisite functionality of Object Gateway introduced in Jewel allows to create multiple zones and mirror data and metadata between them. Sync Modules are built atop of the multisite framework that allows for forwarding data and metadata to a different external tier. A sync module allows for a set of actions to be performed whenever a change in data occurs (metadata ops like bucket or user creation etc. are also regarded as changes in data). As the rgw multisite changes are eventually consistent at remote sites, changes are propagated asynchronously. This would allow for unlocking use cases such as backing up the object storage to an external cloud cluster or a custom backup solution using tape drives, indexing metadata in Elasticsearch etc.

13.7.1 Synchronizing Zones #

A sync module configuration is local to a zone. The sync module determines whether the zone exports data or can only consume data that was modified in another zone. As of luminous the supported sync plug-ins are elasticsearch, rgw, which is the default sync plug-in that synchronizes data between the zones and log which is a trivial sync plug-in that logs the metadata operation that happens in the remote zones. The following sections are written with the example of a zone using elasticsearch sync module. The process would be similar for configuring any other sync plug-in.

Note: Default Sync Plugin

rgw is the default sync plug-in and there is no need to explicitly configure this.

13.7.1.1 Requirements and Assumptions #

Let us assume a simple multisite configuration as described in Section 13.11, “Multisite Object Gateways” consisting of the 2 zones us-east and us-west. Now we add a third zone us-east-es which is a zone that only processes metadata from the other sites. This zone can be in the same or a different Ceph cluster than us-east. This zone would only consume metadata from other zones and Object Gateways in this zone will not serve any end user requests directly.

13.7.1.2 Configuring Sync Modules #

Create the third zone similar to the ones described in Section 13.11, “Multisite Object Gateways”, for example

cephadm > radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-east-es \
--access-key={system-key} --secret={secret} --endpoints=http://rgw-es:80

A sync module can be configured for this zone via the following

cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --tier-type={tier-type} \
--tier-config={set of key=value pairs}

For example in the elasticsearch sync module

cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --tier-type=elasticsearch \
--tier-config=endpoint=http://localhost:9200,num_shards=10,num_replicas=1

For the various supported tier-config options refer to Section 13.7.2, “Storing Metadata in Elasticsearch”.

Finally update the period

cephadm > radosgw-admin period update --commit

Now start the radosgw in the zone

root # systemctl start ceph-radosgw@rgw.`hostname -s`
root # systemctl enable ceph-radosgw@rgw.`hostname -s`

13.7.2 Storing Metadata in Elasticsearch #

This sync module writes the metadata from other zones to Elasticsearch. As of luminous this is JSON of data fields we currently store in Elasticsearch.

{
  "_index" : "rgw-gold-ee5863d6",
  "_type" : "object",
  "_id" : "34137443-8592-48d9-8ca7-160255d52ade.34137.1:object1:null",
  "_score" : 1.0,
  "_source" : {
    "bucket" : "testbucket123",
    "name" : "object1",
    "instance" : "null",
    "versioned_epoch" : 0,
    "owner" : {
      "id" : "user1",
      "display_name" : "user1"
    },
    "permissions" : [
      "user1"
    ],
    "meta" : {
      "size" : 712354,
      "mtime" : "2017-05-04T12:54:16.462Z",
      "etag" : "7ac66c0f148de9519b8bd264312c4d64"
    }
  }
}

13.7.2.1 Elasticsearch Tier Type Configuration Parameters #

endpoint: Specifies the Elasticsearch server endpoint to access.
num_shards: (integer) The number of shards that Elasticsearch will be configured with on data sync initialization. Note that this cannot be changed after initialization. Any change here requires rebuild of the Elasticsearch index and reinitialization of the data sync process.
num_replicas: (integer) The number of the replicas that Elasticsearch will be configured with on data sync initialization.
explicit_custom_meta: (true | false) Specifies whether all user custom metadata will be indexed, or whether user will need to configure (at the bucket level) what customer metadata entries should be indexed. This is false by default
index_buckets_list: (comma separated list of strings) If empty, all buckets will be indexed. Otherwise, only buckets specified here will be indexed. It is possible to provide bucket prefixes (for example 'foo*'), or bucket suffixes (for example '*bar').
approved_owners_list: (comma separated list of strings) If empty, buckets of all owners will be indexed (subject to other restrictions), otherwise, only buckets owned by specified owners will be indexed. Suffixes and prefixes can also be provided.
override_index_path: (string) if not empty, this string will be used as the Elasticsearch index path. Otherwise the index path will be determined and generated on sync initialization.

13.7.2.2 Metadata Queries #

Since the Elasticsearch cluster now stores object metadata, it is important that the Elasticsearch endpoint is not exposed to the public and only accessible to the cluster administrators. For exposing metadata queries to the end user itself this poses a problem since we'd want the user to only query their metadata and not of any other users, this would require the Elasticsearch cluster to authenticate users in a way similar to RGW does which poses a problem.

As of Luminous RGW in the metadata master zone can now service end user requests. This allows for not exposing the Elasticsearch endpoint in public and also solves the authentication and authorization problem since RGW itself can authenticate the end user requests. For this purpose RGW introduces a new query in the bucket APIs that can service Elasticsearch requests. All these requests must be sent to the metadata master zone.

Get an Elasticsearch Query

GET /BUCKET?query={query-expr}

request params:

max-keys: max number of entries to return
marker: pagination marker

expression := [(]<arg> <op> <value> [)][<and|or> ...]

op is one of the following: <, <=, ==, >=, >

For example:

GET /?query=name==foo

Will return all the indexed keys that user has read permission to, and are named 'foo'. The output will be a list of keys in XML that is similar to the S3 list buckets response.

Configure custom metadata fields

Define which custom metadata entries should be indexed (under the specified bucket), and what are the types of these keys. If explicit custom metadata indexing is configured, this is needed so that rgw will index the specified custom metadata values. Otherwise it is needed in cases where the indexed metadata keys are of a type other than string.

POST /BUCKET?mdsearch
x-amz-meta-search: <key [; type]> [, ...]

Multiple metadata fields must be comma separated, a type can be forced for a field with a `;`. The currently allowed types are string(default), integer and date, for example, if you want to index a custom object metadata x-amz-meta-year as int, x-amz-meta-date as type date and x-amz-meta-title as string, you would do

POST /mybooks?mdsearch
x-amz-meta-search: x-amz-meta-year;int, x-amz-meta-release-date;date, x-amz-meta-title;string

Delete custom metadata configuration

Delete custom metadata bucket configuration.

DELETE /BUCKET?mdsearch

Get custom metadata configuration

Retrieve custom metadata bucket configuration.

GET /BUCKET?mdsearch

13.8 LDAP Authentication #

Apart from the default local user authentication, Object Gateway can use LDAP server services to authenticate users as well.

13.8.1 Authentication Mechanism #

The Object Gateway extracts the user's LDAP credentials from a token. A search filter is constructed from the user name. The Object Gateway uses the configured service account to search the directory for a matching entry. If an entry is found, the Object Gateway attempts to bind to the found distinguished name with the password from the token. If the credentials are valid, the bind will succeed, and the Object Gateway grants access.

You can limit the allowed users by setting the base for the search to a specific organizational unit or by specifying a custom search filter, for example requiring specific group membership, custom object classes, or attributes.

13.8.2 Requirements #

LDAP or Active Directory: A running LDAP instance accessible by the Object Gateway.
Service account: LDAP credentials to be used by the Object Gateway with search permissions.
User account: At least one user account in the LDAP directory.

Important: Do Not Overlap LDAP and Local Users

You should not use the same user names for local users and for users being authenticated by using LDAP. The Object Gateway cannot distinguish them and it treats them as the same user.

Tip: Sanity Checks

Use the ldapsearch utility to verify the service account or the LDAP connection. For example:

cephadm > ldapsearch -x -D "uid=ceph,ou=system,dc=example,dc=com" -W \
-H ldaps://example.com -b "ou=users,dc=example,dc=com" 'uid=*' dn

Make sure to use the same LDAP parameters as in the Ceph configuration file to eliminate possible problems.

13.8.3 Configure Object Gateway to Use LDAP Authentication #

The following parameters in the /etc/ceph/ceph.conf configuration file are related to the LDAP authentication:

rgw_ldap_uri: Specifies the LDAP server to use. Make sure to use the ldaps://fqdn:port parameter to avoid transmitting the plain text credentials openly.
rgw_ldap_binddn: The Distinguished Name (DN) of the service account used by the Object Gateway.
rgw_ldap_secret: The password for the service account.
rgw_ldap_searchdn: Specifies the base in the directory information tree for searching users. This might be your users organizational unit or some more specific Organizational Unit (OU).
rgw_ldap_dnattr: The attribute being used in the constructed search filter to match a user name. Depending on your Directory Information Tree (DIT) this would probably be uid or cn.
rgw_search_filter: If not specified, the Object Gateway automatically constructs the search filter with the rgw_ldap_dnattr setting. Use this parameter to narrow the list of allowed users in very flexible ways. Consult Section 13.8.4, “Using a Custom Search Filter to Limit User Access” for details.

13.8.4 Using a Custom Search Filter to Limit User Access #

There are two ways you can use the rgw_search_filter parameter.

13.8.4.1 Partial Filter to Further Limit the Constructed Search Filter #

An example of a partial filter:

"objectclass=inetorgperson"

The Object Gateway will generate the search filter as usual with the user name from the token and the value of rgw_ldap_dnattr. The constructed filter is then combined with the partial filter from the rgw_search_filter attribute. Depending on the user name and the settings the final search filter may become:

"(&(uid=hari)(objectclass=inetorgperson))"

In that case, user 'hari' will only be granted access if he is found in the LDAP directory, has an object class of 'inetorgperson', and did specify a valid password.

13.8.4.2 Complete Filter #

A complete filter must contain a USERNAME token which will be substituted with the user name during the authentication attempt. The rgw_ldap_dnattr parameter is not used anymore in this case. For example, to limit valid users to a specific group, use the following filter:

"(&(uid=USERNAME)(memberOf=cn=ceph-users,ou=groups,dc=mycompany,dc=com))"

Note: `memberOf` Attribute

Using the memberOf attribute in LDAP searches requires server side support from you specific LDAP server implementation.

13.8.5 Generating an Access Token for LDAP authentication #

The radosgw-token utility generates the access token based on the LDAP user name and password. It outputs a base-64 encoded string which is the actual access token. Use your favorite S3 client (refer to Section 13.5.1, “Accessing Object Gateway”) and specify the token as the access key and use an empty secret key.

cephadm > export RGW_ACCESS_KEY_ID="username"
cephadm > export RGW_SECRET_ACCESS_KEY="password"
cephadm > radosgw-token --encode --ttype=ldap

Important: Clear Text Credentials

The access token is a base-64 encoded JSON structure and contains the LDAP credentials as a clear text.

Note: Active Directory

For Active Directory, use the --ttype=ad parameter.

13.9 Bucket Index Sharding #

The Object Gateway stores bucket index data in an index pool, which defaults to .rgw.buckets.index. If you put too many (hundreds of thousands) objects into a single bucket and the quota for maximum number of objects per bucket (rgw bucket default quota max objects) is not set, the performance of the index pool may degrade. Bucket index sharding prevents such performance decreases and allows a high number of objects per bucket.

13.9.1 Bucket Index Resharding #

If a bucket has grown large and its initial configuration is not sufficient anymore, the bucket's index pool needs to be resharded. You can either use automatic online bucket index resharding (refer to Section 13.9.1.1, “Dynamic Resharding”, or reshard the bucket index offline manually (refer to Section 13.9.1.2, “Manual Resharding”.

13.9.1.1 Dynamic Resharding #

Since SUSE Enterprise Storage 5.5, we support online bucket resharding. It detects if the number of objects per bucket reaches a certain threshold, and automatically increases the number of shards used by the bucket index. This process reduces the number of entries in each bucket index shard.

The detection process runs:

When new objects are added to the bucket.
In a background process that periodically scans all the buckets. This is needed in order to deal with existing buckets that are not being updated.

A bucket that requires resharding is added to the reshard_log queue and will be scheduled to be resharded later. The reshard threads run in the background and execute the scheduled resharding, one at a time.

Configuring Dynamic Resharding #

rgw_dynamic_resharding: Enables or disables dynamic bucket index resharding. Possible values are 'true' or 'false'. Defaults to 'true'.
rgw_reshard_num_logs: Number of shards for the resharding log. Defaults to 16.
rgw_reshard_bucket_lock_duration: Duration of lock on the bucket object during resharding. Defaults to 120 seconds.
rgw_max_objs_per_shard: Maximum number of objects per bucket index shard. Defaults to 100000 objects.
rgw_reshard_thread_interval: Maximum time between rounds of reshard thread processing. Defaults to 600 seconds.

Important: Multisite Configurations

Dynamic resharding is not supported in multisite environment. It is disabled by default since Ceph 12.2.2, but we recommend you to double check the setting.

Commands to Administer the Resharding Process #

Add a bucket to the resharding queue:

cephadm > radosgw-admin reshard add \
 --bucket BUCKET_NAME \
 --num-shards NEW_NUMBER_OF_SHARDS

List resharding queue:

cephadm > radosgw-admin reshard list

Process / schedule a bucket resharding:

cephadm > radosgw-admin reshard process

Display the bucket resharding status:

cephadm > radosgw-admin reshard status --bucket BUCKET_NAME

Cancel pending bucket resharding:

cephadm > radosgw-admin reshard cancel --bucket BUCKET_NAME

13.9.1.2 Manual Resharding #

Dynamic resharding mentioned in Section 13.9.1.1, “Dynamic Resharding” is supported only for simple Object Gateway configurations. For multisite configurations, use manual resharding described in this section.

To reshard the bucket index manually offline, use the following command:

cephadm > radosgw-admin bucket reshard

The bucket reshard command performs the following:

Creates a new set of bucket index objects for the specified object.
Spreads all objects entries of these index objects.
Creates a new bucket instance.
Links the new bucket instance with the bucket so that all new index operations go through the new bucket indexes.
Prints the old and the new bucket ID to the standard output.

Procedure 13.1: Resharding the Bucket Index Pool #

Make sure that all operations to the bucket are stopped.

Back up the original bucket index:

cephadm > radosgw-admin bi list \
 --bucket=BUCKET_NAME \
 > BUCKET_NAME.list.backup

Reshard the bucket index:
```
 cephadm > radosgw-admin reshard \
 --bucket=BUCKET_NAME \
 --num-shards=NEW_SHARDS_NUMBER
```
Tip: Old Bucket ID
As part of its output, this command also prints the new and the old bucket ID. Note the old bucket ID down; you will need it to purge the old bucket index objects.
Verify that the objects are listed correctly by comparing the old bucket index listing with the new one. Then purge the old bucket index objects:
```
cephadm > radosgw-admin bi purge
 --bucket=BUCKET_NAME
 --bucket-id=OLD_BUCKET_ID
```

13.9.2 Bucket Index Sharding for New Buckets #

There are two options that affect bucket index sharding:

Use the rgw_override_bucket_index_max_shards option for simple configurations.
Use the bucket_index_max_shards option for multisite configurations.

Setting the options to 0 disables bucket index sharding. A value greater than 0 enables bucket index sharding and sets the maximum number of shards.

The following formula helps you calculate the recommended number of shards:

number_of_objects_expected_in_a_bucket / 100000

Be aware that the maximum number of shards is 7877.

13.9.2.1 Simple Configurations #

Open the Ceph configuration file and add or modify the following option:
```
rgw_override_bucket_index_max_shards = 12
```
Tip: All or One Object Gateway Instances
To configure bucket index sharding for all instances of the Object Gateway, include rgw_override_bucket_index_max_shards in the [global] section.
To configure bucket index sharding only for a particular instance of the Object Gateway, include rgw_override_bucket_index_max_shards in the related instance section.
Restart the Object Gateway. See Section 13.3, “Operating the Object Gateway Service” for more details.

13.9.2.2 Multisite Configurations #

Multisite configurations can have a different index pool to manage failover. To configure a consistent shard count for zones in one zone group, set the bucket_index_max_shards option in the zone group's configuration:

Export the zone group configuration to the zonegroup.json file:
```
cephadm > radosgw-admin zonegroup get > zonegroup.json
```
Edit the zonegroup.json file and set the bucket_index_max_shards option for each named zone.

Reset the zone group:

cephadm > radosgw-admin zonegroup set < zonegroup.json

Update the period:

cephadm > radosgw-admin period update --commit

13.10 Integrating OpenStack Keystone #

OpenStack Keystone is an identity service for the OpenStack product. You can integrate the Object Gateway with Keystone to set up a gateway that accepts a Keystone authentication token. A user authorized by Keystone to access the gateway will be verified on the Ceph Object Gateway side and automatically created if needed. The Object Gateway queries Keystone periodically for a list of revoked tokens.

13.10.1 Configuring OpenStack #

Before configuring the Ceph Object Gateway, you need to configure the OpenStack Keystone to enable the Swift service and point it to the Ceph Object Gateway:

Set the Swift service. To use OpenStack to validate Swift users, first create the Swift service:
```
cephadm > openstack service create \
 --name=swift \
 --description="Swift Service" \
 object-store
```

Set the endpoints. After you create the Swift service, point to the Ceph Object Gateway. Replace REGION_NAME with the name of the gateway’s zone group name or region name.

cephadm > openstack endpoint create --region REGION_NAME \
 --publicurl   "http://radosgw.example.com:8080/swift/v1" \
 --adminurl    "http://radosgw.example.com:8080/swift/v1" \
 --internalurl "http://radosgw.example.com:8080/swift/v1" \
 swift

Verify the settings. After you create the Swift service and set the endpoints, show the endpoints to verify that all the settings are correct.
```
cephadm > openstack endpoint show object-store
```

13.10.2 Configuring the Ceph Object Gateway #

13.10.2.1 Configure SSL Certificates #

The Ceph Object Gateway queries Keystone periodically for a list of revoked tokens. These requests are encoded and signed. Keystone may be also configured to provide self-signed tokens, which are also encoded and signed. You need to configure the gateway so that it can decode and verify these signed messages. Therefore, the OpenSSL certificates that Keystone uses to create the requests need to be converted to the 'nss db' format:

root # mkdir /var/ceph/nss
root # openssl x509 -in /etc/keystone/ssl/certs/ca.pem \
 -pubkey | certutil -d /var/ceph/nss -A -n ca -t "TCu,Cu,Tuw"
rootopenssl x509 -in /etc/keystone/ssl/certs/signing_cert.pem \
 -pubkey | certutil -A -d /var/ceph/nss -n signing_cert -t "P,P,P"

To allow Ceph Object Gateway to interact with OpenStack Keystone, OpenStack Keystone can use a self-signed SSL certificate. Either install Keystone’s SSL certificate on the node running the Ceph Object Gateway, or alternatively set the value of the option rgw keystone verify ssl to 'false'. Setting rgw keystone verify ssl to 'false' means that the gateway will not attempt to verify the certificate.

13.10.2.2 Configure the Object Gateway's Options #

You can configure Keystone integration using the following options:

rgw keystone api version: Version of the Keystone API. Valid options are 2 or 3. Defaults to 2.
rgw keystone url: The URL and port number of the administrative RESTful API on the Keystone server. Follows the pattern SERVER_URL:PORT_NUMBER.
rgw keystone admin token: The token or shared secret that is configured internally in Keystone for administrative requests.
rgw keystone accepted roles: The roles required to serve requests. Defaults to 'Member, admin'.
rgw keystone accepted admin roles: The list of roles allowing a user to gain administrative privileges.
rgw keystone token cache size: The maximum number of entries in the Keystone token cache.
rgw keystone revocation interval: The number of seconds before checking revoked tokens. Defaults to 15 * 60.
rgw keystone implicit tenants: Create new users in their own tenants of the same name. Defaults to 'false'.
rgw s3 auth use keystone: If set to 'true', the Ceph Object Gateway will authenticate users using Keystone. Defaults to 'false'.
nss db path: The path to the NSS database.

It is also possible to configure the Keystone service tenant, user & password for keystone (for v2.0 version of the OpenStack Identity API), similar to the way OpenStack services tend to be configured. This way you can avoid setting the shared secret rgw keystone admin token in the configuration file, which should be disabled in production environments. The service tenant credentials should have admin privileges, for more details refer to the official OpenStack Keystone documentation. The related configuration options follow:

rgw keystone admin user: The Keystone administrator user name.
rgw keystone admin password: The keystone administrator user password.
rgw keystone admin tenant: The Keystone version 2.0 administrator user tenant.

A Ceph Object Gateway user is mapped to a Keystone tenant. A Keystone user has different roles assigned to it, possibly on more than a single tenant. When the Ceph Object Gateway gets the ticket, it looks at the tenant and the user roles that are assigned to that ticket, and accepts or rejects the request according to the setting of the rgw keystone accepted roles option.

Tip: Mapping to OpenStack Tenants

Although Swift tenants are mapped to the Object Gateway user by default, they can be also mapped to OpenStack tenants via the rgw keystone implicit tenants option. This will make containers use the tenant namespace instead of the S3 like global namespace that the Object Gateway defaults to. We recommend deciding on the mapping method at the planning stage to avoid confusion. The reason is that toggling the option later affects only newer requests which get mapped under a tenant, while older buckets created before still continue to be in a global namespace.

For version 3 of the OpenStack Identity API, you should replace the rgw keystone admin tenant option with:

rgw keystone admin domain: The Keystone administrator user domain.
rgw keystone admin project: The Keystone administrator user project.

13.11 Multisite Object Gateways #

Zone

A logical grouping of one or more Object Gateway instances. There must be one zone designated as the master zone in a zonegroup, which handles all bucket and user creation.

Zonegroup

A zonegroup consists of multiple zones. There should be a master zonegroup that will handle changes to the system configuration.

Zonegroup map

A configuration structure that holds the map of the entire system, for example which zonegroup is the master, relationships between different zone groups, and certain configuration options such as storage policies.

Realm

A container for zone groups. This allows for separation of zone groups between clusters. It is possible to create multiple realms, making it easier to run completely different configurations in the same cluster.

Period

A period holds the configuration structure for the current state of the realm. Every period contains a unique ID and an epoch. Every realm has an associated current period, holding the current state of configuration of the zone groups and storage policies. Any configuration change for a non-master zone will increment the period's epoch. Changing the master zone to a different zone will trigger the following changes:

A new period is generated with a new period ID and epoch of 1.
Realm's current period is updated to point to the newly generated period ID.
Realm's epoch is incremented.

You can configure each Object Gateway to participate in a federated architecture, working in an active zone configuration while allowing for writes to non-master zones.

13.11.1 Terminology #

A description of terms specific to a federated architecture follows:

13.11.2 Example Cluster Setup #

In this example, we will focus on creating a single zone group with three separate zones, which actively synchronize their data. Two zones belong to the same cluster, while the third belongs to a different one. There is no synchronization agent involved in mirroring data changes between the Object Gateways. This allows for a much simpler configuration scheme and active-active configurations. Note that metadata operations—such as creating a new user—still need to go through the master zone. However, data operations—such as creation of buckets and objects—can be handled by any of the zones.

13.11.3 System Keys #

While configuring zones, Object Gateway expects creation of an S3-compatible system user together with their access and secret keys. This allows another Object Gateway instance to pull the configuration remotely with the access and secret keys. For more information on creating S3 users, see Section 13.5.2.1, “Adding S3 and Swift Users”.

Tip

It is useful to generate the access and secret keys before the zone creation itself because it makes scripting and use of configuration management tools easier later on.

For the purpose of this example, let us assume that the access and secret keys are set in the environment variables:

# SYSTEM_ACCESS_KEY=1555b35654ad1656d805
# SYSTEM_SECRET_KEY=h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==

Generally, access keys consist of 20 alphanumeric characters, while secret keys consist of 40 alphanumeric characters (they can contain +/= characters as well). You can generate these keys in the command line:

# SYSTEM_ACCESS_KEY=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 20 | head -n 1)
# SYSTEM_SECRET_KEY=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 40 | head -n 1)

13.11.4 Naming Conventions #

This example describes the process of setting up a master zone. We will assume a zonegroup called us spanning the United States, which will be our master zonegroup. This will contain two zones written in a zonegroup-zone format. This is our convention only and you can choose a format that you prefer. In summary:

Master zonegroup: United States us
Master zone: United States, East Region 1: us-east-1
Secondary zone: United States, East Region 2: us-east-2
Secondary zone: United States, West Region: us-west

This will be a part of a larger realm named gold. The us-east-1 and us-east-2 zones are part of the same Ceph cluster, us-east-1 being the primary one. us-west is in a different Ceph cluster.

13.11.5 Default Pools #

When configured with the appropriate permissions, Object Gateway creates default pools on its own. The pg_num and pgp_num values are taken from the ceph.conf configuration file. Pools related to a zone by default follow the convention of zone-name.pool-name. For example for the us-east-1 zone, it will be the following pools:

.rgw.root
us-east-1.rgw.control
us-east-1.rgw.data.root
us-east-1.rgw.gc
us-east-1.rgw.log
us-east-1.rgw.intent-log
us-east-1.rgw.usage
us-east-1.rgw.users.keys
us-east-1.rgw.users.email
us-east-1.rgw.users.swift
us-east-1.rgw.users.uid
us-east-1.rgw.buckets.index
us-east-1.rgw.buckets.data
us-east-1.rgw.meta

These pools can be created in other zones as well, by replacing us-east-1 with the appropriate zone name.

13.11.6 Creating a Realm #

Configure a realm called gold and make it the default realm:

cephadm > radosgw-admin realm create --rgw-realm=gold --default
{
  "id": "4a367026-bd8f-40ee-b486-8212482ddcd7",
  "name": "gold",
  "current_period": "09559832-67a4-4101-8b3f-10dfcd6b2707",
  "epoch": 1
}

Note that every realm has an ID, which allows for flexibility such as renaming the realm later if needed. The current_period changes whenever we change anything in the master zone. The epoch is incremented when there is a change in the master zone's configuration which results in a change of the current period.

13.11.7 Deleting the Default Zonegroup #

The default installation of Object Gateway creates the default zonegroup called default. Because we no longer need the default zonegroup, remove it.

cephadm > radosgw-admin zonegroup delete --rgw-zonegroup=default

13.11.8 Creating a Master Zonegroup #

Create a master zonegroup called us. The zonegroup will manage the zonegroup map and propagate changes to the rest of the system. By marking the zonegroup as default, you allow explicitly mentioning the rgw-zonegroup switch for later commands.

cephadm > radosgw-admin zonegroup create --rgw-zonegroup=us \
--endpoints=http://rgw1:80 --master --default
{
  "id": "d4018b8d-8c0d-4072-8919-608726fa369e",
  "name": "us",
  "api_name": "us",
  "is_master": "true",
  "endpoints": [
      "http:\/\/rgw1:80"
  ],
  "hostnames": [],
  "hostnames_s3website": [],
  "master_zone": "",
  "zones": [],
  "placement_targets": [],
  "default_placement": "",
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
}

Alternatively, you can mark a zonegroup as default with the following command:

cephadm > radosgw-admin zonegroup default --rgw-zonegroup=us

13.11.9 Creating a Master Zone #

Now create a default zone and add it to the default zonegroup. Note that you will use this zone for metadata operations such as user creation:

cephadm > radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-east-1 \
--endpoints=http://rgw1:80 --access-key=$SYSTEM_ACCESS_KEY --secret=$SYSTEM_SECRET_KEY
{
  "id": "83859a9a-9901-4f00-aa6d-285c777e10f0",
  "name": "us-east-1",
  "domain_root": "us-east-1/gc.rgw.data.root",
  "control_pool": "us-east-1/gc.rgw.control",
  "gc_pool": "us-east-1/gc.rgw.gc",
  "log_pool": "us-east-1/gc.rgw.log",
  "intent_log_pool": "us-east-1/gc.rgw.intent-log",
  "usage_log_pool": "us-east-1/gc.rgw.usage",
  "user_keys_pool": "us-east-1/gc.rgw.users.keys",
  "user_email_pool": "us-east-1/gc.rgw.users.email",
  "user_swift_pool": "us-east-1/gc.rgw.users.swift",
  "user_uid_pool": "us-east-1/gc.rgw.users.uid",
  "system_key": {
      "access_key": "1555b35654ad1656d804",
      "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r\/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
  },
  "placement_pools": [
      {
          "key": "default-placement",
          "val": {
              "index_pool": "us-east-1/gc.rgw.buckets.index",
              "data_pool": "us-east-1/gc.rgw.buckets.data",
              "data_extra_pool": "us-east-1/gc.rgw.buckets.non-ec",
              "index_type": 0
          }
      }
  ],
  "metadata_heap": "us-east-1/gc.rgw.meta",
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
}

Note that the --rgw-zonegroup and --default switches add the zone to a zonegroup and make it the default zone. Alternatively, the same can also be done with the following commands:

cephadm > radosgw-admin zone default --rgw-zone=us-east-1
cephadm > radosgw-admin zonegroup add --rgw-zonegroup=us --rgw-zone=us-east-1

13.11.9.1 Creating System Users #

To access zone pools, you need to create a system user. Note that you will need these keys when configuring the secondary zone as well.

cephadm > radosgw-admin user create --uid=zone.user \
--display-name="Zone User" --access-key=$SYSTEM_ACCESS_KEY \
--secret=$SYSTEM_SECRET_KEY --system

13.11.9.2 Update the Period #

Because you changed the master zone configuration, you need to commit the changes for them to take effect in the realm configuration structure. Initially, the period looks like this:

cephadm > radosgw-admin period get
{
  "id": "09559832-67a4-4101-8b3f-10dfcd6b2707", "epoch": 1, "predecessor_uuid": "", "sync_status": [], "period_map":
  {
    "id": "09559832-67a4-4101-8b3f-10dfcd6b2707", "zonegroups": [], "short_zone_ids": []
  }, "master_zonegroup": "", "master_zone": "", "period_config":
  {
     "bucket_quota": {
     "enabled": false, "max_size_kb": -1, "max_objects": -1
     }, "user_quota": {
       "enabled": false, "max_size_kb": -1, "max_objects": -1
     }
  }, "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7", "realm_name": "gold", "realm_epoch": 1
}

Update the period and commit the changes:

cephadm > radosgw-admin period update --commit
{
  "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
  "epoch": 1,
  "predecessor_uuid": "09559832-67a4-4101-8b3f-10dfcd6b2707",
  "sync_status": [ "[...]"
  ],
  "period_map": {
      "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
      "zonegroups": [
          {
              "id": "d4018b8d-8c0d-4072-8919-608726fa369e",
              "name": "us",
              "api_name": "us",
              "is_master": "true",
              "endpoints": [
                  "http:\/\/rgw1:80"
              ],
              "hostnames": [],
              "hostnames_s3website": [],
              "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "zones": [
                  {
                      "id": "83859a9a-9901-4f00-aa6d-285c777e10f0",
                      "name": "us-east-1",
                      "endpoints": [
                          "http:\/\/rgw1:80"
                      ],
                      "log_meta": "true",
                      "log_data": "false",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  }
              ],
              "placement_targets": [
                  {
                      "name": "default-placement",
                      "tags": []
                  }
              ],
              "default_placement": "default-placement",
              "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
          }
      ],
      "short_zone_ids": [
          {
              "key": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "val": 630926044
          }
      ]
  },
  "master_zonegroup": "d4018b8d-8c0d-4072-8919-608726fa369e",
  "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
  "period_config": {
      "bucket_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      },
      "user_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      }
  },
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7",
  "realm_name": "gold",
  "realm_epoch": 2
}

13.11.9.3 Start the Object Gateway #

You need to mention the Object Gateway zone and port options in the configuration file before starting the Object Gateway. For more information on Object Gateway and its configuration, see Chapter 13, Ceph Object Gateway. The configuration section of Object Gateway should look similar to this:

[client.rgw.us-east-1]
rgw_frontends="civetweb port=80"
rgw_zone=us-east-1

Start the Object Gateway:

root # systemctl start ceph-radosgw@rgw.us-east-1

13.11.10 Creating a Secondary Zone #

In the same cluster, create and configure the secondary zone named us-east-2. You can execute all the following commands in the node hosting the master zone itself.

To create the secondary zone, use the same command as when you created the primary zone, except dropping the master flag:

cephadm > radosgw-admin zone create --rgw-zonegroup=us --endpoints=http://rgw2:80 \
--rgw-zone=us-east-2 --access-key=$SYSTEM_ACCESS_KEY --secret=$SYSTEM_SECRET_KEY
{
  "id": "950c1a43-6836-41a2-a161-64777e07e8b8",
  "name": "us-east-2",
  "domain_root": "us-east-2.rgw.data.root",
  "control_pool": "us-east-2.rgw.control",
  "gc_pool": "us-east-2.rgw.gc",
  "log_pool": "us-east-2.rgw.log",
  "intent_log_pool": "us-east-2.rgw.intent-log",
  "usage_log_pool": "us-east-2.rgw.usage",
  "user_keys_pool": "us-east-2.rgw.users.keys",
  "user_email_pool": "us-east-2.rgw.users.email",
  "user_swift_pool": "us-east-2.rgw.users.swift",
  "user_uid_pool": "us-east-2.rgw.users.uid",
  "system_key": {
      "access_key": "1555b35654ad1656d804",
      "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r\/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
  },
  "placement_pools": [
      {
          "key": "default-placement",
          "val": {
              "index_pool": "us-east-2.rgw.buckets.index",
              "data_pool": "us-east-2.rgw.buckets.data",
              "data_extra_pool": "us-east-2.rgw.buckets.non-ec",
              "index_type": 0
          }
      }
  ],
  "metadata_heap": "us-east-2.rgw.meta",
  "realm_id": "815d74c2-80d6-4e63-8cfc-232037f7ff5c"
}

13.11.10.1 Update the Period #

Inform all the gateways of the new change in the system map by doing a period update and committing the changes:

cephadm > radosgw-admin period update --commit
{
  "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
  "epoch": 2,
  "predecessor_uuid": "09559832-67a4-4101-8b3f-10dfcd6b2707",
  "sync_status": [ "[...]"
  ],
  "period_map": {
      "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
      "zonegroups": [
          {
              "id": "d4018b8d-8c0d-4072-8919-608726fa369e",
              "name": "us",
              "api_name": "us",
              "is_master": "true",
              "endpoints": [
                  "http:\/\/rgw1:80"
              ],
              "hostnames": [],
              "hostnames_s3website": [],
              "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "zones": [
                  {
                      "id": "83859a9a-9901-4f00-aa6d-285c777e10f0",
                      "name": "us-east-1",
                      "endpoints": [
                          "http:\/\/rgw1:80"
                      ],
                      "log_meta": "true",
                      "log_data": "false",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  },
                  {
                      "id": "950c1a43-6836-41a2-a161-64777e07e8b8",
                      "name": "us-east-2",
                      "endpoints": [
                          "http:\/\/rgw2:80"
                      ],
                      "log_meta": "false",
                      "log_data": "true",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  }

              ],
              "placement_targets": [
                  {
                      "name": "default-placement",
                      "tags": []
                  }
              ],
              "default_placement": "default-placement",
              "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
          }
      ],
      "short_zone_ids": [
          {
              "key": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "val": 630926044
          },
          {
              "key": "950c1a43-6836-41a2-a161-64777e07e8b8",
              "val": 4276257543
          }

      ]
  },
  "master_zonegroup": "d4018b8d-8c0d-4072-8919-608726fa369e",
  "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
  "period_config": {
      "bucket_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      },
      "user_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      }
  },
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7",
  "realm_name": "gold",
  "realm_epoch": 2
}

13.11.10.2 Start the Object Gateway #

Adjust the configuration of the Object Gateway for the secondary zone, and start it:

[client.rgw.us-east-2]
rgw_frontends="civetweb port=80"
rgw_zone=us-east-2

cephadm > sudo systemctl start ceph-radosgw@rgw.us-east-2

13.11.11 Adding Object Gateway to the Second Cluster #

The second Ceph cluster belongs to the same zonegroup as the initial one, but may be geographically located elsewhere.

13.11.11.1 Default Realm and Zonegroup #

Since you already created the realm for the first gateway, pull the realm here and make it the default here:

cephadm > radosgw-admin realm pull --url=http://rgw1:80 \
--access-key=$SYSTEM_ACCESS_KEY --secret=$SYSTEM_SECRET_KEY
{
  "id": "4a367026-bd8f-40ee-b486-8212482ddcd7",
  "name": "gold",
  "current_period": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
  "epoch": 2
}
cephadm > radosgw-admin realm default --rgw-realm=gold

Get the configuration from the master zone by pulling the period:

cephadm > radosgw-admin period pull --url=http://rgw1:80 \
--access-key=$SYSTEM_ACCESS_KEY --secret=$SYSTEM_SECRET_KEY

Set the default zonegroup to the already created us zonegroup:

cephadm > radosgw-admin zonegroup default --rgw-zonegroup=us

13.11.11.2 Secondary Zone Configuration #

Create a new zone named us-west with the same system keys:

cephadm > radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-west \
--access-key=$SYSTEM_ACCESS_KEY --secret=$SYSTEM_SECRET_KEY \
--endpoints=http://rgw3:80 --default
{
  "id": "950c1a43-6836-41a2-a161-64777e07e8b8",
  "name": "us-west",
  "domain_root": "us-west.rgw.data.root",
  "control_pool": "us-west.rgw.control",
  "gc_pool": "us-west.rgw.gc",
  "log_pool": "us-west.rgw.log",
  "intent_log_pool": "us-west.rgw.intent-log",
  "usage_log_pool": "us-west.rgw.usage",
  "user_keys_pool": "us-west.rgw.users.keys",
  "user_email_pool": "us-west.rgw.users.email",
  "user_swift_pool": "us-west.rgw.users.swift",
  "user_uid_pool": "us-west.rgw.users.uid",
  "system_key": {
      "access_key": "1555b35654ad1656d804",
      "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r\/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
  },
  "placement_pools": [
      {
          "key": "default-placement",
          "val": {
              "index_pool": "us-west.rgw.buckets.index",
              "data_pool": "us-west.rgw.buckets.data",
              "data_extra_pool": "us-west.rgw.buckets.non-ec",
              "index_type": 0
          }
      }
  ],
  "metadata_heap": "us-west.rgw.meta",
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
}

13.11.11.3 Update the Period #

To propagate the zonegroup map changes, we update and commit the period:

cephadm > radosgw-admin period update --commit --rgw-zone=us-west
{
  "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
  "epoch": 3,
  "predecessor_uuid": "09559832-67a4-4101-8b3f-10dfcd6b2707",
  "sync_status": [
      "", # truncated
  ],
  "period_map": {
      "id": "b5e4d3ec-2a62-4746-b479-4b2bc14b27d1",
      "zonegroups": [
          {
              "id": "d4018b8d-8c0d-4072-8919-608726fa369e",
              "name": "us",
              "api_name": "us",
              "is_master": "true",
              "endpoints": [
                  "http:\/\/rgw1:80"
              ],
              "hostnames": [],
              "hostnames_s3website": [],
              "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "zones": [
                  {
                      "id": "83859a9a-9901-4f00-aa6d-285c777e10f0",
                      "name": "us-east-1",
                      "endpoints": [
                          "http:\/\/rgw1:80"
                      ],
                      "log_meta": "true",
                      "log_data": "true",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  },
                                  {
                      "id": "950c1a43-6836-41a2-a161-64777e07e8b8",
                      "name": "us-east-2",
                      "endpoints": [
                          "http:\/\/rgw2:80"
                      ],
                      "log_meta": "false",
                      "log_data": "true",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  },
                  {
                      "id": "d9522067-cb7b-4129-8751-591e45815b16",
                      "name": "us-west",
                      "endpoints": [
                          "http:\/\/rgw3:80"
                      ],
                      "log_meta": "false",
                      "log_data": "true",
                      "bucket_index_max_shards": 0,
                      "read_only": "false"
                  }
              ],
              "placement_targets": [
                  {
                      "name": "default-placement",
                      "tags": []
                  }
              ],
              "default_placement": "default-placement",
              "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7"
          }
      ],
      "short_zone_ids": [
          {
              "key": "83859a9a-9901-4f00-aa6d-285c777e10f0",
              "val": 630926044
          },
          {
              "key": "950c1a43-6836-41a2-a161-64777e07e8b8",
              "val": 4276257543
          },
          {
              "key": "d9522067-cb7b-4129-8751-591e45815b16",
              "val": 329470157
          }
      ]
  },
  "master_zonegroup": "d4018b8d-8c0d-4072-8919-608726fa369e",
  "master_zone": "83859a9a-9901-4f00-aa6d-285c777e10f0",
  "period_config": {
      "bucket_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      },
      "user_quota": {
          "enabled": false,
          "max_size_kb": -1,
          "max_objects": -1
      }
  },
  "realm_id": "4a367026-bd8f-40ee-b486-8212482ddcd7",
  "realm_name": "gold",
  "realm_epoch": 2
}

Note that the period epoch number has incremented, indicating a change in the configuration.

13.11.11.4 Start the Object Gateway #

This is similar to starting the Object Gateway in the first zone. The only difference is that the Object Gateway zone configuration should reflect the us-west zone name:

[client.rgw.us-west]
rgw_frontends="civetweb port=80"
rgw_zone=us-west

Start the second Object Gateway:

root # systemctl start ceph-radosgw@rgw.us-west

13.11.12 Failover and Disaster Recovery #

If the master zone should fail, failover to the secondary zone for disaster recovery.

Make the secondary zone the master and default zone. For example:
```
cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --master --default
```
By default, Ceph Object Gateway will run in an active-active configuration. If the cluster was configured to run in an active-passive configuration, the secondary zone is a read-only zone. Remove the --read-only status to allow the zone to receive write operations. For example:
```
cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --master --default \
--read-only=False
```
Update the period to make the changes take effect.
```
cephadm > radosgw-admin period update --commit
```

Finally, restart the Ceph Object Gateway.

root # systemctl restart ceph-radosgw@rgw.`hostname -s`

If the former master zone recovers, revert the operation.

From the recovered zone, pull the period from the current master zone.

cephadm > radosgw-admin period pull --url={url-to-master-zone-gateway} \
--access-key={access-key} --secret={secret}

Make the recovered zone the master and default zone.

cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --master --default

Update the period to make the changes take effect.
```
cephadm > radosgw-admin period update --commit
```
Then, restart the Ceph Object Gateway in the recovered zone.
```
root # systemctl restart ceph-radosgw@rgw.`hostname -s`
```
If the secondary zone needs to be a read-only configuration, update the secondary zone.
```
cephadm > radosgw-admin zone modify --rgw-zone={zone-name} --read-only
```
Update the period to make the changes take effect.
```
cephadm > radosgw-admin period update --commit
```
Finally, restart the Ceph Object Gateway in the secondary zone.
```
root # systemctl restart ceph-radosgw@rgw.`hostname -s`
```

13.12 Load Balancing the Object Gateway Servers with HAProxy #

You can use the HAProxy load balancer to distribute all requests across multiple Object Gateway back-end servers. Refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-lb-haproxy for more details on configuring HAProxy.

Following is a simple configuration of HAProxy for balancing Object Gateway nodes using round robin as the balancing algorithm:

cephadm > cat /etc/haproxy/haproxy.cfg
[...]
frontend https_frontend
bind *:443 crt path-to-cert.pem [ciphers: ... ]
default_backend rgw

backend rgw
mode http
balance roundrobin
server rgw_server1 rgw-endpoint1 weight 1 maxconn 100 check
server rgw_server2 rgw-endpoint2 weight 1 maxconn 100 check
[...]

13 Ceph Object Gateway #

13.1 Object Gateway Restrictions and Naming Limitations #

13.1.1 Bucket Limitations #

Tip: Use DNS-compliant Bucket Names

13.1.2 Stored Object Limitations #

13.1.3 HTTP Header Limitations #

13.2 Deploying the Object Gateway #

13.3 Operating the Object Gateway Service #

13.4 Configuration Parameters #

General Settings #

Note: tcp_nodelay

Multisite Settings #

Swift Settings #

Warning

Logging Settings #

Keystone Settings #

13.4.1 Additional Notes #

13.5 Managing Object Gateway Access #

13.5.1 Accessing Object Gateway #

13.5.1.1 S3 Interface Access #

13.5.1.2 Swift Interface Access #

13.5.2 Managing S3 and Swift Accounts #

13.5.2.1 Adding S3 and Swift Users #

13.5.2.2 Removing S3 and Swift Users #

Tip: Removing a Subuser

13.5.2.3 Changing S3 and Swift User Access and Secret Keys #

13.5.2.4 User Quota Management #

13.6 Enabling HTTPS/SSL for Object Gateways #

13.6.1 Create a Self-Signed Certificate #

Note

Tip: IP Addresses in subjectAltName

13.6.2 Simple HTTPS Configuration #

13.6.3 Advanced HTTPS Configuration #

Tip: Binding to Multiple Ports

13.7 Sync Modules #

13.7.1 Synchronizing Zones #

Note: Default Sync Plugin

13.7.1.1 Requirements and Assumptions #

13.7.1.2 Configuring Sync Modules #

13.7.2 Storing Metadata in Elasticsearch #

13.7.2.1 Elasticsearch Tier Type Configuration Parameters #

13.7.2.2 Metadata Queries #

13.8 LDAP Authentication #

13.8.1 Authentication Mechanism #

13.8.2 Requirements #

Important: Do Not Overlap LDAP and Local Users

Tip: Sanity Checks

13.8.3 Configure Object Gateway to Use LDAP Authentication #

13.8.4 Using a Custom Search Filter to Limit User Access #

13.8.4.1 Partial Filter to Further Limit the Constructed Search Filter #

13.8.4.2 Complete Filter #

Note: memberOf Attribute

13.8.5 Generating an Access Token for LDAP authentication #

Important: Clear Text Credentials

Note: Active Directory

13.9 Bucket Index Sharding #

13.9.1 Bucket Index Resharding #

13.9.1.1 Dynamic Resharding #

Configuring Dynamic Resharding #

Important: Multisite Configurations

Commands to Administer the Resharding Process #

13.9.1.2 Manual Resharding #

Procedure 13.1: Resharding the Bucket Index Pool #

Tip: Old Bucket ID

13.9.2 Bucket Index Sharding for New Buckets #

13.9.2.1 Simple Configurations #

Tip: All or One Object Gateway Instances

13.9.2.2 Multisite Configurations #

13.10 Integrating OpenStack Keystone #

13.10.1 Configuring OpenStack #

13.10.2 Configuring the Ceph Object Gateway #

13.10.2.1 Configure SSL Certificates #

13.10.2.2 Configure the Object Gateway's Options #

Tip: Mapping to OpenStack Tenants

13.11 Multisite Object Gateways #

13.11.1 Terminology #

13.11.2 Example Cluster Setup #

13.11.3 System Keys #

Tip

13.11.4 Naming Conventions #

Note: `tcp_nodelay`

Tip: IP Addresses in `subjectAltName`

Note: `memberOf` Attribute