14 Frequently asked questions #
14.1 How does the number of placement groups affect the cluster performance? #
When your cluster is becoming 70% to 80% full, it is time to add more OSDs to it. When you increase the number of OSDs, you may consider increasing the number of placement groups as well.
Changing the number of PGs causes a lot of data transfer within the cluster.
To calculate the optimal value for your newly-resized cluster is a complex task.
A high number of PGs creates small chunks of data. This speeds up recovery after an OSD failure, but puts a lot of load on the monitor nodes as they are responsible for calculating the data location.
On the other hand, a low number of PGs takes more time and data transfer to recover from an OSD failure, but does not impose that much load on monitor nodes as they need to calculate locations for less (but larger) data chunks.
Find more information on the optimal number of PGs for your cluster in Section 17.4.2, “Determining the value of PG_NUM”.
14.2 Can I use SSDs and hard disks on the same cluster? #
Solid-state drives (SSD) are generally faster than hard disks. If you mix the two types of disks for the same write operation, the data writing to the SSD disk will be slowed down by the hard disk performance. Thus, you should never mix SSDs and hard disks for data writing following the same rule (see Section 17.3, “Rule sets” for more information on rules for storing data).
There are generally 2 cases where using SSD and hard disk on the same cluster makes sense:
Use each disk type for writing data following different rules. Then you need to have a separate rule for the SSD disk, and another rule for the hard disk.
Use each disk type for a specific purpose. For example the SSD disk for journal, and the hard disk for storing data.
14.3 What are the trade-offs of ssing a journal on SSD? #
Using SSDs for OSD journal(s) is better for performance as the journal is usually the bottleneck of hard disk-only OSDs. SSDs are often used to share journals of several OSDs.
Following is a list of potential disadvantages of using SSDs for OSD journal:
SSD disks are more expensive than hard disks. But as one OSD journal requires up to 6GB of disk space only, the price may not be so crucial.
SSD disk consumes storage slots which can be otherwise used by a large hard disk to extend the cluster capacity.
SSD disks have reduced write cycles compared to hard disks, but modern technologies are beginning to eliminate the problem.
If you share more journals on the same SSD disk, you risk losing all the related OSDs after the SSD disk fails. This will require a lot of data to be moved to rebalance the cluster.
Hotplugging disks becomes more complex as the data mapping is not 1:1 the failed OSD and the journal disk.
14.4 What happens when a disk fails? #
When a disk with a stored cluster data has a hardware problem and fails to operate, here is what happens:
The related OSD crashed and is automatically removed from the cluster.
The failed disk's data is replicated to another OSD in the cluster from other copies of the same data stored in other OSDs.
Then you should remove the disk from the cluster CRUSH Map, and physically from the host hardware.
14.5 What happens when a journal disk fails? #
Ceph can be configured to store journals or write ahead logs on devices separate from the OSDs. When a disk dedicated to a journal fails, the related OSD(s) fail as well (see Section 14.4, “What happens when a disk fails?”).
For performance boost, you can use a fast disk (such as SSD) to store journal partitions for several OSDs. We do not recommend to host journals for more than 4 OSDs on one disk, because in case of the journals' disk failure, you risk losing stored data for all the related OSDs' disks.