Adding a new OSD (Jewel + CentOS 7.2)

This procedure is suitable for adding either a whole disk device or a single partition of a disk.

Prepare the device

Disks or partitions which have previously been a component of a RAID set or which have contained a filesystem should be cleared. The following commands will do the trick depending on what the disk or partition was used for:

sgdisk -Z /dev/{disk-or-partition}; partprobe

mdadm --zero-superblock /dev/{disk-or-partition}

dd if=/dev/zero of=/dev/{disk-or-partition} bs=4096 count=10240


Perform as root on the host where the OSD will live.

Create a new logical OSD:

ceph osd create

The above will return the number of the new OSD.

Create the mount point for the new OSD:

mkdir /var/lib/ceph/osd/ceph-{osd-number}

Format the disk or partition and mount it:

mkfs -t xfs /dev/{disk-or-partition}
mount -t xfs -o noatime,inode64 /dev/{disk-or-partition} /var/lib/ceph/osd/ceph-{osd-number}

Initialize the top-level directory of the OSD:

ceph-osd -i {osd-number} --mkfs --mkkey

Register the OSD's key:

ceph auth add osd.{osd-number} osd 'allow *' mon 'allow rwx' -i /var/lib/ceph/osd/ceph-{osd-number}/keyring

Add the OSD to the CRUSH map:

ceph osd crush add {osd-number} {weight} host={host}

It can be a good idea to start with the {weight} set to 0.0 tp prevent premature migration starting.

{host} is the name of the host on which the OSD lives. This determines where the OSD appears in the OSD tree.

Set ownership of OSD:

chown -R ceph:ceph /var/lib/ceph/osd/ceph-{osd-number}

Get the OSD filesystem UUID:

blkid /dev/{disk-or-partition}

Add the following to /etc/fstab to ensure the OSD mounts at boot time:

UUID={uuid} /var/lib/ceph/osd/ceph-{osd-number} xfs noatime,inode64 1 2

Create the file nice.conf in /usr/lib/systemd/system/ceph-osd\@.service.d/:


This ensures that the OSD daemon processes get timely access to CPU and I/O resources.

Start the OSD daemon:

systemctl start ceph-osd\@{osd-number}
systemctl enable ceph-osd\@{osd-number}

See also:

Adding/Removing OSDs @