24 Export Ceph data via Samba #

Make sure that a working CephFS already exists in your cluster.

Create a Samba Gateway specific keyring on the Ceph admin node and copy it to both Samba Gateway nodes:

cephuser@adm > ceph auth get-or-create client.samba.gw mon 'allow r' \
 osd 'allow *' mds 'allow *' -o ceph.client.samba.gw.keyring
cephuser@adm > scp ceph.client.samba.gw.keyring SAMBA_NODE:/etc/ceph/

Replace SAMBA_NODE with the name of the Samba gateway node.

The following steps are executed on the Samba Gateway node. Install Samba together with the Ceph integration package:

cephuser@smb > sudo zypper in samba samba-ceph

Replace the default contents of the /etc/samba/smb.conf file with the following:

[global]
  netbios name = SAMBA-GW
  clustering = no
  idmap config * : backend = tdb2
  passdb backend = tdbsam
  # disable print server
  load printers = no
  smbd: backgroundqueue = no

[SHARE_NAME]
  path = CEPHFS_MOUNT
  read only = no
  oplocks = no
  kernel share modes = no

The CEPHFS_MOUNT path above must be mounted prior to starting Samba with a kernel CephFS share configuration. See Section 23.3, “Mounting CephFS in /etc/fstab”.

The above share configuration uses the Linux kernel CephFS client, which is recommended for performance reasons. As an alternative, the Samba vfs_ceph module can also be used to communicate with the Ceph cluster. The instructions are shown below for legacy purposes and are not recommended for new Samba deployments:

[SHARE_NAME]
  path = /
  vfs objects = ceph
  ceph: config_file = /etc/ceph/ceph.conf
  ceph: user_id = samba.gw
  read only = no
  oplocks = no
  kernel share modes = no

Tip: Oplocks and share modes

oplocks (also known as SMB2+ leases) allow for improved performance through aggressive client caching, but are currently unsafe when Samba is deployed together with other CephFS clients, such as kernel mount.ceph, FUSE, or NFS Ganesha.

If all CephFS file system path access is exclusively handled by Samba, then the oplocks parameter can be safely enabled.

Currently kernel share modes needs to be disabled in a share running with the CephFS vfs module for file serving to work properly.

Important: Permitting access

Samba maps SMB users and groups to local accounts. Local users can be assigned a password for Samba share access via:

# smbpasswd -a USERNAME

For successful I/O, the share path's access control list (ACL) needs to permit access to the user connected via Samba. You can modify the ACL by temporarily mounting via the CephFS kernel client and using the chmod, chown, or setfacl utilities against the share path. For example, to permit access for all users, run:

# chmod 777 MOUNTED_SHARE_PATH

Create a Samba Gateway specific keyring on the Admin Node and copy it to both nodes:

cephuser@adm > ceph auth get-or-create client.samba.gw mon 'allow r' \
    osd 'allow *' mds 'allow *' -o ceph.client.samba.gw.keyring
cephuser@adm > scp ceph.client.samba.gw.keyring earth:/etc/ceph/
cephuser@adm > scp ceph.client.samba.gw.keyring mars:/etc/ceph/

SLE-HA setup requires a fencing device to avoid a split brain situation when active cluster nodes become unsynchronized. For this purpose, you can use a Ceph RBD image with Stonith Block Device (SBD). Refer to https://documentation.suse.com/sle-ha/15-SP3/html/SLE-HA-all/cha-ha-storage-protect.html#sec-ha-storage-protect-fencing-setup for more details.

If it does not yet exist, create an RBD pool called rbd (see Section 18.1, “Creating a pool”) and associate it with rbd (see Section 18.5.1, “Associating pools with an application”). Then create a related RBD image called sbd01:

cephuser@adm > ceph osd pool create rbd
cephuser@adm > ceph osd pool application enable rbd rbd
cephuser@adm > rbd -p rbd create sbd01 --size 64M --image-shared

Prepare earth and mars to host the Samba service:

1. Make sure the following packages are installed before you proceed: ctdb, tdb-tools, and samba.

   # zypper in ctdb tdb-tools samba samba-ceph

2. Make sure the Samba and CTDB services are stopped and disabled:

   # systemctl disable ctdb
   # systemctl disable smb
   # systemctl disable nmb
   # systemctl disable winbind
   # systemctl stop ctdb
   # systemctl stop smb
   # systemctl stop nmb
   # systemctl stop winbind

3. Open port 4379 of your firewall on all nodes. This is needed for CTDB to communicate with other cluster nodes.

On earth, create the configuration files for Samba. They will later automatically synchronize to mars.

1. Insert a list of private IP addresses of Samba Gateway nodes in the /etc/ctdb/nodes file. Find more details in the ctdb manual page (man 7 ctdb).

   192.168.1.1
   192.168.1.2

2. Configure Samba. Add the following lines in the [global] section of /etc/samba/smb.conf. Use the host name of your choice in place of CTDB-SERVER (all nodes in the cluster will appear as one big node with this name). Add a share definition as well, consider SHARE_NAME as an example:

   [global]
     netbios name = SAMBA-HA-GW
     clustering = yes
     idmap config * : backend = tdb2
     passdb backend = tdbsam
     ctdbd socket = /var/lib/ctdb/ctdb.socket
     # disable print server
     load printers = no
     smbd: backgroundqueue = no
   
   [SHARE_NAME]
     path = /
     vfs objects = ceph
     ceph: config_file = /etc/ceph/ceph.conf
     ceph: user_id = samba.gw
     read only = no
     oplocks = no
     kernel share modes = no

   Note that the /etc/ctdb/nodes and /etc/samba/smb.conf files need to match on all Samba Gateway nodes.

Install and bootstrap the SUSE Linux Enterprise High Availability cluster.

1. Register the SUSE Linux Enterprise High Availability Extension on earth and mars:

   root@earth # SUSEConnect -r ACTIVATION_CODE -e E_MAIL

   root@mars # SUSEConnect -r ACTIVATION_CODE -e E_MAIL

2. Install ha-cluster-bootstrap on both nodes:

   root@earth # zypper in ha-cluster-bootstrap

   root@mars # zypper in ha-cluster-bootstrap

3. Map the RBD image sbd01 on both Samba Gateways via rbdmap.service.

   Edit /etc/ceph/rbdmap and add an entry for the SBD image:

   rbd/sbd01 id=samba.gw,keyring=/etc/ceph/ceph.client.samba.gw.keyring

   Enable and start rbdmap.service:

   root@earth # systemctl enable rbdmap.service && systemctl start rbdmap.service
   root@mars # systemctl enable rbdmap.service && systemctl start rbdmap.service

   The /dev/rbd/rbd/sbd01 device should be available on both Samba Gateways.

4. Initialize the cluster on earth and let mars join it.

   root@earth # ha-cluster-init

   root@mars # ha-cluster-join -c earth

   

   Important

   During the process of initialization and joining the cluster, you will be interactively asked whether to use SBD. Confirm with y and then specify /dev/rbd/rbd/sbd01 as a path to the storage device.

Check the status of the cluster. You should see two nodes added in the cluster:

root@earth # crm status
2 nodes configured
1 resource configured

Online: [ earth mars ]

Full list of resources:

 admin-ip       (ocf::heartbeat:IPaddr2):       Started earth

Execute the following commands on earth to configure the CTDB resource:

root@earth # crm configure
crm(live)configure# primitive ctdb ocf:heartbeat:CTDB params \
    ctdb_manages_winbind="false" \
    ctdb_manages_samba="false" \
    ctdb_recovery_lock="!/usr/lib64/ctdb/ctdb_mutex_ceph_rados_helper
        ceph client.samba.gw cephfs_metadata ctdb-mutex"
    ctdb_socket="/var/lib/ctdb/ctdb.socket" \
        op monitor interval="10" timeout="20" \
        op start interval="0" timeout="200" \
        op stop interval="0" timeout="100"
crm(live)configure# primitive smb systemd:smb \
    op start timeout="100" interval="0" \
    op stop timeout="100" interval="0" \
    op monitor interval="60" timeout="100"
crm(live)configure# primitive nmb systemd:nmb \
    op start timeout="100" interval="0" \
    op stop timeout="100" interval="0" \
    op monitor interval="60" timeout="100"
crm(live)configure# primitive winbind systemd:winbind \
    op start timeout="100" interval="0" \
    op stop timeout="100" interval="0" \
    op monitor interval="60" timeout="100"
crm(live)configure# group g-ctdb ctdb winbind nmb smb
crm(live)configure# clone cl-ctdb g-ctdb meta interleave="true"
crm(live)configure# commit

Tip: Optional nmb and winbind primitives

If you do not require network share browsing, you do not need to add the nmb primitive.

The winbind primitive is only needed when configured as an Active Directory domain member. See Section 24.2, “Joining Samba Gateway and Active Directory”.

The binary /usr/lib64/ctdb/ctdb_mutex_ceph_rados_helper in the configuration option ctdb_recovery_lock has the parameters CLUSTER_NAME, CEPHX_USER, RADOS_POOL, and RADOS_OBJECT, in this order.

An extra lock-timeout parameter can be appended to override the default value used (10 seconds). A higher value will increase the CTDB recovery master failover time, whereas a lower value may result in the recovery master being incorrectly detected as down, triggering flapping failovers.

Add a clustered IP address:

crm(live)configure# primitive ip ocf:heartbeat:IPaddr2
    params ip=192.168.2.1 \
    unique_clone_address="true" \
    op monitor interval="60" \
    meta resource-stickiness="0"
crm(live)configure# clone cl-ip ip \
    meta interleave="true" clone-node-max="2" globally-unique="true"
crm(live)configure# colocation col-with-ctdb 0: cl-ip cl-ctdb
crm(live)configure# order o-with-ctdb 0: cl-ip cl-ctdb
crm(live)configure# commit

If unique_clone_address is set to true, the IPaddr2 resource agent adds a clone ID to the specified address, leading to three different IP addresses. These are usually not needed, but help with load balancing. For further information about this topic, see https://documentation.suse.com/sle-ha/15-SP3/html/SLE-HA-all/cha-ha-lb.html.

Check the result:

root@earth # crm status
Clone Set: base-clone [dlm]
     Started: [ factory-1 ]
     Stopped: [ factory-0 ]
 Clone Set: cl-ctdb [g-ctdb]
     Started: [ factory-1 ]
     Started: [ factory-0 ]
 Clone Set: cl-ip [ip] (unique)
     ip:0       (ocf:heartbeat:IPaddr2):       Started factory-0
     ip:1       (ocf:heartbeat:IPaddr2):       Started factory-1

Test from a client machine. On a Linux client, run the following command to see if you can copy files from and to the system:

# smbclient //192.168.2.1/myshare