Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE Enterprise Storage 5.5 (SES 5 & SES 5.5)

13 Exporting Ceph Data via Samba Edit source

This chapter describes how to export data stored in a Ceph cluster via a Samba/CIFS share so that you can easily access them from Windows* client machines. It also includes information that will help you configure a Ceph Samba gateway to join Active Directory in the Windows* domain to authenticate and authorize users.

Note
Note: Samba Gateway Performance

Because of increased protocol overhead and additional latency caused by extra network hops between the client and the storage, accessing CephFS via a Samba Gateway may significantly reduce application performance when compared to native Ceph clients.

13.1 Export CephFS via Samba Share Edit source

Warning
Warning: Cross Protocol Access

Native CephFS and NFS clients are not restricted by file locks obtained via Samba, and vice versa. Applications that rely on cross protocol file locking may experience data corruption if CephFS backed Samba share paths are accessed via other means.

13.1.1 Samba Related Packages Installation Edit source

To configure and export a Samba share, the following packages need to be installed: samba-ceph and samba-winbind. If these packages are not installed, install them:

cephadm@smb > zypper install samba-ceph samba-winbind

13.1.2 Single Gateway Example Edit source

In preparation for exporting a Samba share, choose an appropriate node to act as a Samba Gateway. The node needs to have access to the Ceph client network, as well as sufficient CPU, memory, and networking resources.

Failover functionality can be provided with CTDB and the SUSE Linux Enterprise High Availability Extension. Refer to Section 13.1.3, “High Availability Configuration” for more information on HA setup.

  1. Make sure that a working CephFS already exists in your cluster. For details, see Chapter 11, Installation of CephFS.

  2. Create a Samba Gateway specific keyring on the Ceph admin node and copy it to both Samba Gateway nodes:

    cephadm > ceph auth get-or-create client.samba.gw mon 'allow r' \
     osd 'allow *' mds 'allow *' -o ceph.client.samba.gw.keyring
    cephadm > scp ceph.client.samba.gw.keyring SAMBA_NODE:/etc/ceph/

    Replace SAMBA_NODE with the name of the Samba gateway node.

  3. The following steps are executed on the Samba Gateway node. Install Samba together with the Ceph integration package:

    cephadm@smb > sudo zypper in samba samba-ceph
  4. Replace the default contents of the /etc/samba/smb.conf file with the following:

    [global]
      netbios name = SAMBA-GW
      clustering = no
      idmap config * : backend = tdb2
      passdb backend = tdbsam
      # disable print server
      load printers = no
      smbd: backgroundqueue = no
    
    [SHARE_NAME]
      path = /
      vfs objects = ceph
      ceph: config_file = /etc/ceph/ceph.conf
      ceph: user_id = samba.gw
      read only = no
      oplocks = no
      kernel share modes = no
    Tip
    Tip: Oplocks and Share Modes

    oplocks (also known as SMB2+ leases) allow for improved performance through aggressive client caching, but are currently unsafe when Samba is deployed together with other CephFS clients, such as kernel mount.ceph, FUSE, or NFS Ganesha.

    Currently kernel share modes needs to be disabled in a share running with the CephFS vfs module for file serving to work properly.

    Important
    Important: Permitting Access

    Since vfs_ceph does not require a file system mount, the share path is interpreted as an absolute path within the Ceph file system on the attached Ceph cluster. For successful share I/O, the path's access control list (ACL) needs to permit access from the mapped user for the given Samba client. You can modify the ACL by temporarily mounting via the CephFS kernel client and using the chmod, chown or setfacl utilities against the share path. For example, to permit access for all users, run:

    root # chmod 777 MOUNTED_SHARE_PATH
  5. Start and enable the Samba daemon:

    cephadm@smb > sudo systemctl start smb.service
    cephadm@smb > sudo systemctl enable smb.service
    cephadm@smb > sudo systemctl start nmb.service
    cephadm@smb > sudo systemctl enable nmb.service

13.1.3 High Availability Configuration Edit source

Important
Important: Transparent Failover Not Supported

Although a multi-node Samba + CTDB deployment is more highly available compared to the single node (see Chapter 13, Exporting Ceph Data via Samba), client-side transparent failover is not supported. Applications will likely experience a short outage on Samba Gateway node failure.

This section provides an example of how to set up a two-node high availability configuration of Samba servers. The setup requires the SUSE Linux Enterprise High Availability Extension. The two nodes are called earth (192.168.1.1) and mars (192.168.1.2).

For details about SUSE Linux Enterprise High Availability Extension, see https://documentation.suse.com/sle-ha/12-SP5/.

Additionally, two floating virtual IP addresses allow clients to connect to the service no matter which physical node it is running on. 192.168.1.10 is used for cluster administration with Hawk2 and 192.168.2.1 is used exclusively for the CIFS exports. This makes it easier to apply security restrictions later.

The following procedure describes the example installation. More details can be found at https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-install-quick/.

  1. Create a Samba Gateway specific keyring on the Admin Node and copy it to both nodes:

    cephadm > ceph auth get-or-create client.samba.gw mon 'allow r' \
        osd 'allow *' mds 'allow *' -o ceph.client.samba.gw.keyring
    cephadm > scp ceph.client.samba.gw.keyring earth:/etc/ceph/
    cephadm > scp ceph.client.samba.gw.keyring mars:/etc/ceph/
  2. SLE-HA setup requires a fencing device to avoid a split brain situation when active cluster nodes become unsynchronized. For this purpose, you can use a Ceph RBD image with Stonith Block Device (SBD). Refer to https://documentation.suse.com/sle-ha/12-SP5/single-html/SLE-HA-guide/#sec-ha-storage-protect-fencing-setup for more details.

    If it does not yet exist, create an RBD pool called rbd (see Section 8.2.2, “Create a Pool”) and associate it with rbd (see Section 8.1, “Associate Pools with an Application”). Then create a related RBD image called sbd01:

    cephadm > ceph osd pool create rbd PG_NUM PGP_NUM replicated
    cephadm > ceph osd pool application enable rbd rbd
    cephadm > rbd -p rbd create sbd01 --size 64M --image-shared
  3. Prepare earth and mars to host the Samba service:

    1. Make sure the following packages are installed before you proceed: ctdb, tdb-tools, and samba (needed for smb.service and nmb.service).

      cephadm@smb > zypper in ctdb tdb-tools samba samba-ceph
    2. Make sure the services ctdb, smb, and nmb are stopped and disabled:

      cephadm@smb > sudo systemctl disable ctdb
      cephadm@smb > sudo systemctl disable smb
      cephadm@smb > sudo systemctl disable nmb
      cephadm@smb > sudo systemctl stop smb
      cephadm@smb > sudo systemctl stop nmb
    3. Open port 4379 of your firewall on all nodes. This is needed for CTDB to communicate with other cluster nodes.

  4. On earth, create the configuration files for Samba. They will later automatically synchronize to mars.

    1. Insert a list of private IP addresses of Samba Gateway nodes in the /etc/ctdb/nodes file. Find more details in the ctdb manual page (man 7 ctdb).

      192.168.1.1
      192.168.1.2
    2. Configure Samba. Add the following lines in the [global] section of /etc/samba/smb.conf. Use the host name of your choice in place of CTDB-SERVER (all nodes in the cluster will appear as one big node with this name). Add a share definition as well, consider SHARE_NAME as an example:

      [global]
        netbios name = SAMBA-HA-GW
        clustering = yes
        idmap config * : backend = tdb2
        passdb backend = tdbsam
        ctdbd socket = /var/lib/ctdb/ctdb.socket
        # disable print server
        load printers = no
        smbd: backgroundqueue = no
      
      [SHARE_NAME]
        path = /
        vfs objects = ceph
        ceph: config_file = /etc/ceph/ceph.conf
        ceph: user_id = samba.gw
        read only = no
        oplocks = no
        kernel share modes = no

      Note that the /etc/ctdb/nodes and /etc/samba/smb.conf files need to match on all Samba Gateway nodes.

  5. Install and bootstrap the SUSE Linux Enterprise High Availability cluster.

    1. Register the SUSE Linux Enterprise High Availability Extension on earth and mars:

      root@earth # SUSEConnect -r ACTIVATION_CODE -e E_MAIL
      root@mars # SUSEConnect -r ACTIVATION_CODE -e E_MAIL
    2. Install ha-cluster-bootstrap on both nodes:

      root@earth # zypper in ha-cluster-bootstrap
      root@mars # zypper in ha-cluster-bootstrap
    3. Map the RBD image sbd01 on both Samba Gateways via rbdmap.service.

      Edit /etc/ceph/rbdmap and add an entry for the SBD image:

      rbd/sbd01 id=samba.gw,keyring=/etc/ceph/ceph.client.samba.gw.keyring

      Enable and start rbdmap.service:

      root@earth # systemctl enable rbdmap.service && systemctl start rbdmap.service
      root@mars # systemctl enable rbdmap.service && systemctl start rbdmap.service

      The /dev/rbd/rbd/sbd01 device should be available on both Samba Gateways.

    4. Initialize the cluster on earth and let mars join it.

      root@earth # ha-cluster-init
      root@mars # ha-cluster-join -c earth
      Important
      Important

      During the process of initialization and joining the cluster, you will be interactively asked whether to use SBD. Confirm with y and then specify /dev/rbd/rbd/sbd01 as a path to the storage device.

  6. Check the status of the cluster. You should see two nodes added in the cluster:

    root@earth # crm status
    2 nodes configured
    1 resource configured
    
    Online: [ earth mars ]
    
    Full list of resources:
    
     admin-ip       (ocf::heartbeat:IPaddr2):       Started earth
  7. Execute the following commands on earth to configure the CTDB resource:

    root@earth # crm configure
    crm(live)configure# primitive ctdb ocf:heartbeat:CTDB params \
        ctdb_manages_winbind="false" \
        ctdb_manages_samba="false" \
        ctdb_recovery_lock="!/usr/lib64/ctdb/ctdb_mutex_ceph_rados_helper
            ceph client.samba.gw cephfs_metadata ctdb-mutex"
        ctdb_socket="/var/lib/ctdb/ctdb.socket" \
            op monitor interval="10" timeout="20" \
            op start interval="0" timeout="200" \
            op stop interval="0" timeout="100"
    crm(live)configure# primitive nmb systemd:nmb \
        op start timeout="100" interval="0" \
        op stop timeout="100" interval="0" \
        op monitor interval="60" timeout="100"
    crm(live)configure# primitive smb systemd:smb \
        op start timeout="100" interval="0" \
        op stop timeout="100" interval="0" \
        op monitor interval="60" timeout="100"
    crm(live)configure# group g-ctdb ctdb nmb smb
    crm(live)configure# clone cl-ctdb g-ctdb meta interleave="true"
    crm(live)configure# commit

    The binary /usr/lib64/ctdb/ctdb_mutex_ceph_rados_helper in the configuration option ctdb_recovery_lock has the parameters CLUSTER_NAME, CEPHX_USER, RADOS_POOL, and RADOS_OBJECT, in this order.

    An extra lock-timeout parameter can be appended to override the default value used (10 seconds). A higher value will increase the CTDB recovery master failover time, whereas a lower value may result in the recovery master being incorrectly detected as down, triggering flapping failovers.

  8. Add a clustered IP address:

    crm(live)configure# primitive ip ocf:heartbeat:IPaddr2 params ip=192.168.2.1 \
        unique_clone_address="true" \
        op monitor interval="60" \
        meta resource-stickiness="0"
    crm(live)configure# clone cl-ip ip \
        meta interleave="true" clone-node-max="2" globally-unique="true"
    crm(live)configure# colocation col-with-ctdb 0: cl-ip cl-ctdb
    crm(live)configure# order o-with-ctdb 0: cl-ip cl-ctdb
    crm(live)configure# commit

    If unique_clone_address is set to true, the IPaddr2 resource agent adds a clone ID to the specified address, leading to three different IP addresses. These are usually not needed, but help with load balancing. For further information about this topic, see https://documentation.suse.com/sle-ha/15-SP1/single-html/SLE-HA-guide/#cha-ha-lb.

  9. Check the result:

    root@earth # crm status
    Clone Set: base-clone [dlm]
         Started: [ factory-1 ]
         Stopped: [ factory-0 ]
     Clone Set: cl-ctdb [g-ctdb]
         Started: [ factory-1 ]
         Started: [ factory-0 ]
     Clone Set: cl-ip [ip] (unique)
         ip:0       (ocf:heartbeat:IPaddr2):       Started factory-0
         ip:1       (ocf:heartbeat:IPaddr2):       Started factory-1
  10. Test from a client machine. On a Linux client, run the following command to see if you can copy files from and to the system:

    root # smbclient //192.168.2.1/myshare
Print this page