Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / SUSE Enterprise Storage 7.1 Documentation / Administration and Operations Guide / Cluster Operation / Operation of Ceph services
Applies to SUSE Enterprise Storage 7.1

14 Operation of Ceph services

You can operate Ceph services on a daemon, node, or cluster level. Depending on which approach you need, use cephadm or the systemctl command.

14.1 Operating individual services

If you need to operate an individual service, identify it first:

cephuser@adm > ceph orch ps
NAME                                 HOST         STATUS         REFRESHED  [...]
mds.my_cephfs.ses-node1.oterul       ses-node1    running (5d)   8m ago
mgr.ses-node1.gpijpm                 ses-node1    running (5d)   8m ago
mgr.ses-node2.oopvyh                 ses-node2    running (5d)   8m ago
mon.ses-node1                        ses-node1    running (5d)   8m ago
mon.ses-node2                        ses-node2    running (5d)   8m ago
mon.ses-node4                        ses-node4    running (5d)   7m ago
osd.0                                ses-node2    running (61m)  8m ago
osd.1                                ses-node3    running (61m)  7m ago
osd.2                                ses-node4    running (61m)  7m ago
rgw.myrealm.myzone.ses-node1.kwwazo  ses-node1    running (5d)   8m ago
rgw.myrealm.myzone.ses-node2.jngabw  ses-node2    error          8m ago

To identify a service on a specific node, run:

ceph orch ps NODE_HOST_NAME

For example:

cephuser@adm > ceph orch ps ses-node2
NAME                                 HOST       STATUS         REFRESHED
mgr.ses-node2.oopvyh                 ses-node2  running (5d)   3m ago
mon.ses-node2                        ses-node2  running (5d)   3m ago
osd.0                                ses-node2  running (67m)  3m ago
Tip
Tip

The ceph orch ps command supports several output formats. To change it, append the --format FORMAT option where FORMAT is one of json, json-pretty, or yaml. For example:

cephuser@adm > ceph orch ps --format yaml

Once you know the name of the service you can start, restart, or stop it:

ceph orch daemon COMMAND SERVICE_NAME

For example, to restart the OSD service with ID 0, run:

cephuser@adm > ceph orch daemon restart osd.0

14.2 Operating service types

If you need to operate a specific type of service across the whole Ceph cluster, use the following command:

ceph orch COMMAND SERVICE_TYPE

Replace COMMAND with either start, stop, or restart.

For example, the following command restarts all MONs in the cluster, regardless of which nodes they actually run on:

cephuser@adm > ceph orch restart mon

14.3 Operating services on a single node

By using the systemctl command, you can operate Ceph related systemd services and targets on a single node.

14.3.1 Identifying services and targets

Before operating Ceph related systemd services and targets, you need to identify the file names of their unit files. File names of the services have the following pattern:

ceph-FSID@SERVICE_TYPE.ID.service

For example:

ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@mon.doc-ses-node1.service
ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@rgw.myrealm.myzone.doc-ses-node1.kwwazo.service
FSID

Unique ID of the Ceph cluster. You can find it in the output of the ceph fsid command.

SERVICE_TYPE

Type of the service, for example osd, mon, or rgw.

ID

Identification string of the service. For OSDs, it is the ID number of the service. For other services, it can be either a host name of the node, or additional strings relevant for the service type.

Tip
Tip

The SERVICE_TYPE.ID part is identical to the content of the NAME column in the output of the ceph orch ps command.

14.3.2 Operating all services on a node

By using Ceph's systemd targets, you can simultaneously operate either all services on a node, or all services that belong to a cluster identified by its FSID.

For example, to stop all Ceph services on a node regardless to which cluster the services belong to, run:

root@minion > systemctl stop ceph.target

To restart all services that belong to a Ceph cluster with ID b4b30c6e-9681-11ea-ac39-525400d7702d, run:

root@minion > systemctl restart ceph-b4b30c6e-9681-11ea-ac39-525400d7702d.target

14.3.3 Operating an individual service on a node

After you have identified the name of a specific service, operate it the following way:

systemctl COMMAND SERVICE_NAME

For example, to restart a single OSD service with ID 1 on a cluster with ID b4b30c6e-9681-11ea-ac39-525400d7702d, run:

# systemctl restart ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@osd.1.service

14.3.4 Querying the service status

You can query systemd for the status of services. For example:

# systemctl status ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@osd.0.service

14.4 Shutting down and restarting the whole Ceph cluster

Shutting down and restarting the cluster may be necessary in the case of a planned power outage. To stop all Ceph related services and restart without issue, follow the steps below.

Procedure 14.1: Shutting down the whole Ceph cluster
  1. Shut down or disconnect any clients accessing the cluster.

  2. To prevent CRUSH from automatically rebalancing the cluster, set the cluster to noout:

    cephuser@adm > ceph osd set noout
  3. Stop all Ceph services on all cluster nodes:

    root@master # ceph-salt stop
  4. Power off all cluster nodes:

    root@master # salt -G 'ceph-salt:member' cmd.run "shutdown -h"
Procedure 14.2: Starting the whole Ceph cluster
  1. Power on the Admin Node.

  2. Power on the Ceph Monitor nodes.

  3. Power on the Ceph OSD nodes.

  4. Unset the previously set noout flag:

    root@master # ceph osd unset noout
  5. Power on all configured gateways.

  6. Power on or connect cluster clients.