14 Operation of Ceph services #
You can operate Ceph services on a daemon, node, or cluster level.
Depending on which approach you need, use cephadm or the
systemctl
command.
14.1 Operating individual services #
If you need to operate an individual service, identify it first:
cephuser@adm >
ceph orch ps
NAME HOST STATUS REFRESHED [...]
mds.my_cephfs.ses-node1.oterul ses-node1 running (5d) 8m ago
mgr.ses-node1.gpijpm ses-node1 running (5d) 8m ago
mgr.ses-node2.oopvyh ses-node2 running (5d) 8m ago
mon.ses-node1 ses-node1 running (5d) 8m ago
mon.ses-node2 ses-node2 running (5d) 8m ago
mon.ses-node4 ses-node4 running (5d) 7m ago
osd.0 ses-node2 running (61m) 8m ago
osd.1 ses-node3 running (61m) 7m ago
osd.2 ses-node4 running (61m) 7m ago
rgw.myrealm.myzone.ses-node1.kwwazo ses-node1 running (5d) 8m ago
rgw.myrealm.myzone.ses-node2.jngabw ses-node2 error 8m ago
To identify a service on a specific node, run:
ceph orch ps NODE_HOST_NAME
For example:
cephuser@adm >
ceph orch ps ses-node2
NAME HOST STATUS REFRESHED
mgr.ses-node2.oopvyh ses-node2 running (5d) 3m ago
mon.ses-node2 ses-node2 running (5d) 3m ago
osd.0 ses-node2 running (67m) 3m ago
The ceph orch ps
command supports several output
formats. To change it, append the --format
FORMAT
option where
FORMAT is one of json
,
json-pretty
, or yaml
. For example:
cephuser@adm >
ceph orch ps --format yaml
Once you know the name of the service you can start, restart, or stop it:
ceph orch daemon COMMAND SERVICE_NAME
For example, to restart the OSD service with ID 0, run:
cephuser@adm >
ceph orch daemon restart osd.0
14.2 Operating service types #
If you need to operate a specific type of service across the whole Ceph cluster, use the following command:
ceph orch COMMAND SERVICE_TYPE
Replace COMMAND with either
start
, stop
, or
restart
.
For example, the following command restarts all MONs in the cluster, regardless of which nodes they actually run on:
cephuser@adm >
ceph orch restart mon
14.3 Operating services on a single node #
By using the systemctl
command, you can operate Ceph
related systemd
services and targets on a single node.
14.3.1 Identifying services and targets #
Before operating Ceph related systemd
services and targets, you need to
identify the file names of their unit files. File names of the services
have the following pattern:
ceph-FSID@SERVICE_TYPE.ID.service
For example:
ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@mon.doc-ses-node1.service
ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@rgw.myrealm.myzone.doc-ses-node1.kwwazo.service
- FSID
Unique ID of the Ceph cluster. You can find it in the output of the
ceph fsid
command.- SERVICE_TYPE
Type of the service, for example
osd
,mon
, orrgw
.- ID
Identification string of the service. For OSDs, it is the ID number of the service. For other services, it can be either a host name of the node, or additional strings relevant for the service type.
The SERVICE_TYPE.ID
part is identical to the content of the NAME
column in
the output of the ceph orch ps
command.
14.3.2 Operating all services on a node #
By using Ceph's systemd
targets, you can simultaneously operate either
all services on a node, or all services that
belong to a cluster identified by its
FSID.
For example, to stop all Ceph services on a node regardless to which cluster the services belong to, run:
root@minion >
systemctl stop ceph.target
To restart all services that belong to a Ceph cluster with ID
b4b30c6e-9681-11ea-ac39-525400d7702d
, run:
root@minion >
systemctl restart ceph-b4b30c6e-9681-11ea-ac39-525400d7702d.target
14.3.3 Operating an individual service on a node #
After you have identified the name of a specific service, operate it the following way:
systemctl COMMAND SERVICE_NAME
For example, to restart a single OSD service with ID 1 on a cluster with ID
b4b30c6e-9681-11ea-ac39-525400d7702d
, run:
#
systemctl restart ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@osd.1.service
14.3.4 Querying the service status #
You can query systemd
for the status of services. For example:
#
systemctl status ceph-b4b30c6e-9681-11ea-ac39-525400d7702d@osd.0.service
14.4 Shutting down and restarting the whole Ceph cluster #
Shutting down and restarting the cluster may be necessary in the case of a planned power outage. To stop all Ceph related services and restart without issue, follow the steps below.
Shut down or disconnect any clients accessing the cluster.
To prevent CRUSH from automatically rebalancing the cluster, set the cluster to
noout
:cephuser@adm >
ceph osd set nooutStop all Ceph services on all cluster nodes:
root@master #
ceph-salt stopPower off all cluster nodes:
root@master #
salt -G 'ceph-salt:member' cmd.run "shutdown -h"
Power on the Admin Node.
Power on the Ceph Monitor nodes.
Power on the Ceph OSD nodes.
Unset the previously set
noout
flag:root@master #
ceph osd unset nooutPower on all configured gateways.
Power on or connect cluster clients.