Applies to SUSE Linux Enterprise Desktop 15 SP5

15 File copying with RSync #

Today, a typical user has several computers: home and workplace machines, a laptop, a smartphone or a tablet. This makes the task of keeping files and documents in synchronization across multiple devices all the more important.

Warning: Risk of data loss

Before you start using a synchronization tool, you should familiarize yourself with its features and functionality. Make sure to back up your important files.

15.1 Conceptual overview #

For synchronizing a large amount of data over a slow network connection, Rsync offers a reliable method of transmitting only changes within files. This applies not only to text files but also binary files. To detect the differences between files, Rsync subdivides the files into blocks and computes check sums over them.

Detecting changes requires certain computing power. So make sure that machines on both ends have enough resources, including RAM.

Rsync can be particularly useful when large amounts of data containing only minor changes need to be transmitted regularly. This is often the case when working with backups. Rsync can also be useful for mirroring staging servers that store complete directory trees of Web servers to a Web server in a DMZ.

Despite its name, Rsync is not a synchronization tool. Rsync is a tool that copies data only in one direction at a time. It does not and cannot do the reverse. If you need a bidirectional tool which can synchronize both source and destination, use Csync.

15.2 Basic syntax #

Rsync is a command-line tool that has the following basic syntax:

rsync [OPTION] SOURCE [SOURCE]... DEST

You can use Rsync on any local or remote machine, provided you have access and write permissions. It is possible to have multiple SOURCE entries. The SOURCE and DEST placeholders can be paths, URLs or both.

Below are the most common Rsync options:

-v: Outputs more verbose text
-a: Archive mode; copies files recursively and preserves time stamps, user/group ownership, file permissions, and symbolic links
-z: Compresses the transmitted data

Note: Trailing slashes count

When working with Rsync, you should pay particular attention to trailing slashes. A trailing slash after the directory denotes the content of the directory. No trailing slash denotes the directory itself.

15.3 Copying files and directories locally #

The following description assumes that the current user has write permissions to the directory /var/backup. To copy a single file from one directory on your machine to another path, use the following command:

> rsync -avz backup.tar.xz /var/backup/

The file backup.tar.xz is copied to /var/backup/; the absolute path will be /var/backup/backup.tar.xz.

Do not forget to add the trailing slash after the /var/backup/ directory. If you do not insert the slash, the file backup.tar.xz is copied to /var/backup (file) not inside the directory /var/backup/!

Copying a directory is similar to copying a single file. The following example copies the directory tux/ and its content into the directory /var/backup/:

> rsync -avz tux /var/backup/

Find the copy in the absolute path /var/backup/tux/.

15.4 Copying files and directories remotely #

The Rsync tool is required on both machines. To copy files from or to remote directories requires an IP address or a domain name. A user name is optional if your current user names on the local and remote machine are the same.

To copy the file file.tar.xz from your local host to the remote host 192.168.1.1 with same users (being local and remote), use the following command:

> rsync -avz file.tar.xz  tux@192.168.1.1:

Depending on what you prefer, these commands are also possible and equivalent:

> rsync -avz file.tar.xz 192.168.1.1:~
> rsync -avz file.tar.xz 192.168.1.1:/home/tux

In all cases with standard configuration, you will be prompted to enter your passphrase of the remote user. This command will copy file.tar.xz to the home directory of user tux (normally /home/tux).

Copying a directory remotely is similar to copying a directory locally. The following example copies the directory tux/ and its content into the remote directory /var/backup/ on the 192.168.1.1 host:

> rsync -avz tux 192.168.1.1:/var/backup/

Assuming you have write permissions on the host 192.168.1.1, you will find the copy in the absolute path /var/backup/tux.

15.5 Configuring and using an rsync server #

Rsync can run as a daemon (rsyncd) listening on default port 873 for incoming connections. This daemon can receive “copying targets”.

The following description explains how to create an Rsync server on a jupiter host with a backup target. This target can be used to store your backups. To create an Rsync server, do the following:

Procedure 15.1: Setting up an rsync server #

On jupiter, create a directory to store all your backup files. In this example, we use /var/backup:
```
# mkdir /var/backup
```
Specify ownership. In this case, the directory is owned by user tux in group users:
```
# chown tux.users /var/backup
```

Configure the rsyncd daemon.

We will separate the configuration file into a main file and certain “modules” which hold your backup target. This makes it easier to add additional targets later. Global values can be stored in /etc/rsyncd.d/*.inc files, whereas your modules are placed in /etc/rsyncd.d/*.conf files:

Create a directory /etc/rsyncd.d/:
```
# mkdir /etc/rsyncd.d/
```

In the main configuration file /etc/rsyncd.conf, add the following lines:

# rsyncd.conf main configuration file
log file = /var/log/rsync.log
pid file = /var/lock/rsync.lock

&merge /etc/rsyncd.d 1
&include /etc/rsyncd.d 2

1	Merges global values from `/etc/rsyncd.d/*.inc` files into the main configuration file.
2	Loads any modules (or targets) from `/etc/rsyncd.d/*.conf` files. These files should not contain any references to global values.

Create your module (your backup target) in the file /etc/rsyncd.d/backup.conf with the following lines:

# backup.conf: backup module
[backup] 1
   uid = tux 2
   gid = users 2
   path = /var/backup 3
   auth users = tux  4
   secrets file = /etc/rsyncd.secrets 5
   comment = Our backup target

1	The backup target. You can use any name you like. However, it is a good idea to name a target according to its purpose and use the same name in your `*.conf` file.
2	Specifies the user name or group name that is used when the file transfer takes place.
3	Defines the path to store your backups (from Step 1).
4	Specifies a comma-separated list of allowed users. In its simplest form, it contains the user names that are allowed to connect to this module. In our case, only user `tux` is allowed.
5	Specifies the path of a file that contains lines with user names and plain passwords.

Create the /etc/rsyncd.secrets file with the following content and replace PASSPHRASE:
```
# user:passwd
tux:PASSPHRASE
```
Make sure the file is only readable by root:
```
# chmod 0600 /etc/rsyncd.secrets
```

Start and enable the rsyncd daemon with:

# systemctl enable rsyncd
# systemctl start rsyncd

Test the access to your Rsync server:
```
> rsync jupiter::
```
You should see a response that looks like this:
```
backup          Our backup target
```
Otherwise, check your configuration file, firewall and network settings.

The above steps create an Rsync server that can now be used to store backups. The example also creates a log file listing all connections. This file is stored in /var/log/rsyncd.log. This is useful to debug your transfers.

To list the content of your backup target, use the following command:

> rsync -avz jupiter::backup

This command lists all files present in the directory /var/backup on the server. This request is also logged in the log file /var/log/rsyncd.log. To start an actual transfer, provide a source directory. Use . for the current directory. For example, the following command copies the current directory to your Rsync backup server:

> rsync -avz . jupiter::backup

By default, Rsync does not delete files and directories when it runs. To enable deletion, the additional option --delete must be stated. To ensure that no newer files are deleted, the option --update can be used instead. Any conflicts that arise must be resolved manually.

15.6 More information #

Csync: Bidirectional file synchronization tool, see https://csync.org/.
RSnapshot: Creates incremental backups, see https://rsnapshot.org.
Unison: A file synchronization tool similar to CSync but with a graphical interface, see https://github.com/bcpierce00/unison.
Rear: A disaster recovery framework, see the Administration Guide of the SUSE Linux Enterprise High Availability, chapter Disaster Recovery with Rear (Relax-and-Recover).