Applies to SUSE Linux Enterprise High Performance Computing 15 SP5

10 Dolly clone tool #

Dolly is used to send data from a management server to many other nodes. It can distribute files, container images, partitions, or whole storage devices.

10.1 Dolly cloning process #

One machine is the management server and distributes the data to the nodes. The management server stores the image, partition, disk, or data to be cloned. This machine runs dolly as a server. All other nodes are dolly clients. They receive the data from the ring, store it locally, and send it to the next node in the ring. All of this happens at the same time, so transferring data to one node or to hundreds of nodes takes the same amount of time.

Dolly creates a virtual TCP ring to distribute data.

Figure 10.1: Dolly cloning process #

10.2 Using dolly #

Install dolly on the management server and all dolly client nodes:

# zypper in dolly

Note: Automatically opened ports

Installing the dolly package automatically opens the TCP ports 9997 and 9998.

The dolly command requires the following information, either directly on the command line or from a configuration file:

Data to send over the network from the dolly server. This could be a storage device, a file (gzip or other file types), or an image (containers or other data).
A target on the dolly clients. The target must be the same data type as the input from the dolly server. For example, dolly cannot send a gzip file to a storage device.
A list of the dolly client nodes where you want to send the data.

Any other parameters are optional. For more information, see man dolly and Section 10.4, “Dolly limitations”.

Procedure 10.1: Cloning data from a dolly server to dolly clients #

On the dolly server, run the following command:

# dolly -s1 -v2 -o LOGFILE3 -I INPUT4 -O OUTPUT5 -H NODE1,NODE2,..6

1	Specifies that this node is the dolly server (the node that sends the data).
2	Switches dolly to verbose mode, which is helpful for debugging.
3	The file that statistical information is written to.
4	The data to clone.
5	The target that will store the data on each dolly client.
6	A comma-separated list of dolly clients to receive the data.

For example, the following command sends /dev/sdc1 to the nodes sle152, sle153, sle154, and sle155:

# dolly -s -v -o /tmp/dolly.log -I /dev/sdc1 -O /dev/sdc1 -H sle152,sle153,sle154,sle155

On each dolly client, start dolly:
```
# dolly -v
```
You can run this command on multiple nodes at once using pdsh. See Section 4.2, “pdsh — parallel remote shell program”.

Example 10.1: Dolly server verbose output #

This example shows typical verbose output from the dolly command on the dolly server:

'writing '192.168.255.2'
'writing '192.168.255.3'
'writing '192.168.255.4'
'writing '192.168.255.5'
'Parameter file:
infile = '/dev/sdc1'
outfile = '/dev/sdc1'
using data port 9998
using ctrl port 9997
myhostname = 'sle151'
fanout = 1
nr_childs = 1
server = 'sle151'
I'm the server.
I'm not the last host.
There are 4 hosts in the ring (excluding server):
        '192.168.255.2'
        '192.168.255.3'
        '192.168.255.4'
        '192.168.255.5'
Next hosts in ring:
        192.168.255.2 (0)
All parameters read successfully.
No compression used.
Using transfer size 4096 bytes.

Trying to build ring...
Connecting to host 192.168.255.2... Send buffer 0 is 131072 bytes
data control.
Waiting for ring to build...
Host got parameters '192.168.255.2'.
Machines left to wait for: 4
Host ready '192.168.255.2'.
Machines left to wait for: 3
Host got parameters '192.168.255.3'.
Machines left to wait for: 3
Host ready '192.168.255.3'.
Machines left to wait for: 2
Host got parameters '192.168.255.4'.
Machines left to wait for: 2
Host ready '192.168.255.4'.
Machines left to wait for: 1
Host got parameters '192.168.255.5'.
Machines left to wait for: 1
Host ready '192.168.255.5'.
Machines left to wait for: 0
Accepted.
Server: Sending data...
Sent MB: 15854, MB/s: 29.655, Current MB/s: 111.111
Read 15854469120 bytes from file(s).
Writing maxbytes = 15854469120 to ctrlout
Sent MB: 15854.
Synced.
Waiting for child 0.
Clients done.
Time: 534.627532
MBytes/s: 29.655
Aggregate MBytes/s: 118.621
Transmitted.

Example 10.2: Dolly client verbose output #

This example shows typical verbose output from the dolly command on the dolly clients:

Trying to build ring...
Buffer size: 98304
Receive buffer is 196608 bytes
Accepting...control...
Trying to read parameters...done.
Parsing parameters...
done.
192.168.255.2 is number 0
Parameter file:
infile = '/dev/sdc1'
outfile = '/dev/sdc1'
using data port 9998
using ctrl port 9997
myhostname = '192.168.255.2'
fanout = 1
nr_childs = 1
server = 'sle151'
I'm not the server.
I'm not the last host.
There are 4 hosts in the ring (excluding server):
        '192.168.255.2'
        '192.168.255.3'
        '192.168.255.4'
        '192.168.255.5'
Next hosts in ring:
192.168.255.3 (1)
All parameters read successfully.
No compression used.
Using transfer size 4096 bytes.
Connected data...done.
Connecting to host 192.168.255.3...
data control.
Accepted.
Receiving...
Transfered MB: 15854, MB/s: 29.655, Current MB/s: 116.661
Max. bytes will be 15854469120 bytes. 49152 bytes left.
Transfered MB: 15854, MB/s: 29.655

Synced.
Transmitted.

10.3 Dolly configuration file #

You can use a dolly configuration file with the -f parameter instead of providing the information manually on the command line. The following example shows a typical configuration file called /etc/dolly.cfg:

infile /tmp/sle15.sif1
outfile /data/sle15.sif2
server sle1513
firstclient sle1524
lastclient sle1545
clients 36
sle1527
sle153
sle154
endconfig8

1	The data to send over the network from the dolly server.
2	The file or device that will store the data on the dolly clients.
3	The name of the dolly server.
4	The first dolly client in the ring.
5	The last dolly client in the ring.
6	Specifies how many dolly clients are in the ring.
7	The list of dolly clients, one per line.
8	Specifies the end of the configuration file.

To use this configuration file, run the following command on the dolly server:

# dolly -v -s -f /etc/dolly.cfg

10.4 Dolly limitations #

Be aware of the following restrictions:

The output data type must be the same as the input data type. Mixing the type of input and output can lead to data corruption.
Only clone partitions that are identical in size on the dolly server and the client node.
Only clone strictly identical storage devices, or corruption can occur.

The following command line parameters are not supported and are provided as a technology preview only:

-S: Ignoring the FQDN is not supported.
-6: Using IPv6 is not supported.
-n: Not doing a sync before exiting is not supported as this can lead to data corruption.
-c: Specifying the uncompressed size of a compressed file should only be used for performance statistics.

The following configuration file options are not supported and are provided as a technology preview only:

compressed: Using the compression option is not supported.
split: Splitting files is not supported (infile or outfile).
fanout: This option must be set to 1 (a linear list). A binary tree or more is not supported.
segsize: This benchmark switch is not supported.
add: Using more than one interface to clone data is not supported.