Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to SUSE Linux Enterprise Server 12 SP5

9 File Copying with RSync Edit source

Today, a typical user has several computers: home and workplace machines, a laptop, a smartphone or a tablet. This makes the task of keeping files and documents in sync across multiple devices all more important.

Warning
Warning: Risk of Data Loss

Before you start using a synchronization tool, you should familiarize yourself with its features and functionality. Make sure to back up your important files.

9.1 Conceptual Overview Edit source

For synchronizing a large amount of data over a slow network connection, Rsync offers a reliable method of transmitting only changes within files. This applies not only to text files but also binary files. To detect the differences between files, Rsync subdivides the files into blocks and computes check sums over them.

Detecting changes requires some computing power. So make sure that machines on both ends have enough resources, including RAM.

Rsync can be particularly useful when large amounts of data containing only minor changes need to be transmitted regularly. This is often the case when working with backups. Rsync can also be useful for mirroring staging servers that store complete directory trees of Web servers to a Web server in a DMZ.

Despite its name, Rsync is not a synchronization tool. Rsync is a tool that copies data only in one direction at a time. It does not and cannot do the reverse. If you need a bidirectional tool which is able to synchronize both source and destination, use Csync.

9.2 Basic Syntax Edit source

Rsync is a command-line tool that has the following basic syntax:

rsync [OPTION] SOURCE [SOURCE]... DEST

You can use Rsync on any local or remote machine, provided you have access and write permissions. It is possible to have multiple SOURCE entries. The SOURCE and DEST placeholders can be paths, URLs, or both.

Below are the most common Rsync options:

-v

Outputs more verbose text

-a

Archive mode; copies files recursively and preserves timestamps, user/group ownership, file permissions, and symbolic links

-z

Compresses the transmitted data

Note
Note: Trailing Slashes Count

When working with Rsync, you should pay particular attention to trailing slashes. A trailing slash after the directory denotes the content of the directory. No trailing slash denotes the directory itself.

9.3 Copying Files and Directories Locally Edit source

The following description assumes that the current user has write permissions to the directory /var/backup. To copy a single file from one directory on your machine to another path, use the following command:

tux > rsync -avz backup.tar.xz /var/backup/

The file backup.tar.xz is copied to /var/backup/; the absolute path will be /var/backup/backup.tar.xz.

Do not forget to add the trailing slash after the /var/backup/ directory! If you do not insert the slash, the file backup.tar.xz is copied to /var/backup (file) not inside the directory /var/backup/!

Copying a directory is similar to copying a single file. The following example copies the directory tux/ and its content into the directory /var/backup/:

tux > rsync -avz tux /var/backup/

Find the copy in the absolute path /var/backup/tux/.

9.4 Copying Files and Directories Remotely Edit source

The Rsync tool is required on both machines. To copy files from or to remote directories requires an IP address or a domain name. A user name is optional if your current user names on the local and remote machine are the same.

To copy the file file.tar.xz from your local host to the remote host 192.168.1.1 with same users (being local and remote), use the following command:

tux > rsync -avz file.tar.xz  tux@192.168.1.1:

Depending on what you prefer, these commands are also possible and equivalent:

tux > rsync -avz file.tar.xz 192.168.1.1:~
tux > rsync -avz file.tar.xz 192.168.1.1:/home/tux

In all cases with standard configuration, you will be prompted to enter your passphrase of the remote user. This command will copy file.tar.xz to the home directory of user tux (usually /home/tux).

Copying a directory remotely is similar to copying a directory locally. The following example copies the directory tux/ and its content into the remote directory /var/backup/ on the 192.168.1.1 host:

tux > rsync -avz tux 192.168.1.1:/var/backup/

Assuming you have write permissions on the host 192.168.1.1, you will find the copy in the absolute path /var/backup/tux.

9.5 Configuring and Using an Rsync Server Edit source

Rsync can run as a daemon (rsyncd) listing on default port 873 for incoming connections. This daemon can receive copying targets.

The following description explains how to create an Rsync server on jupiter with a backup target. This target can be used to store your backups. To create an Rsync server, do the following:

Procedure 9.1: Setting Up an Rsync Server
  1. On jupiter, create a directory to store all your backup files. In this example, we use /var/backup:

    root # mkdir /var/backup
  2. Specify ownership. In this case, the directory is owned by user tux in group users:

    root # chown tux.users /var/backup
  3. Configure the rsyncd daemon.

    We will separate the configuration file into a main file and some modules which hold your backup target. This makes it easier to add additional targets later. Global values can be stored in /etc/rsyncd.d/*.inc files, whereas your modules are placed in /etc/rsyncd.d/*.conf files:

    1. Create a directory /etc/rsyncd.d/:

      root # mkdir /etc/rsyncd.d/
    2. In the main configuration file /etc/rsyncd.conf, add the following lines:

      # rsyncd.conf main configuration file
      log file = /var/log/rsync.log
      pid file = /var/lock/rsync.lock
      
      &merge /etc/rsyncd.d 1
      &include /etc/rsyncd.d 2

      1

      Merges global values from /etc/rsyncd.d/*.inc files into the main configuration file.

      2

      Loads any modules (or targets) from /etc/rsyncd.d/*.conf files. These files should not contain any references to global values.

    3. Create your module (your backup target) in the file /etc/rsyncd.d/backup.conf with the following lines:

      # backup.conf: backup module
      [backup] 1
         uid = tux 2
         gid = users 2
         path = /var/backup 3
         auth users = tux  4
         secrets file = /etc/rsyncd.secrets 5
         comment = Our backup target

      1

      The backup target. You can use any name you like. However, it is a good idea to name a target according to its purpose and use the same name in your *.conf file.

      2

      Specifies the user name or group name that is used when the file transfer takes place.

      3

      Defines the path to store your backups (from Step 1).

      4

      Specifies a comma-separated list of allowed users. In its simplest form, it contains the user names that are allowed to connect to this module. In our case, only user tux is allowed.

      5

      Specifies the path of a file that contains lines with user names and plain passwords.

    4. Create the /etc/rsyncd.secrets file with the following content and replace PASSPHRASE:

      # user:passwd
      tux:PASSPHRASE
    5. Make sure the file is only readable by root:

      root # chmod 0600 /etc/rsyncd.secrets
  4. Start and enable the rsyncd daemon with:

    root # systemctl enable rsyncd
    root # systemctl start rsyncd
  5. Test the access to your Rsync server:

    tux > rsync jupiter::

    You should see a response that looks like this:

    backup          Our backup target

    Otherwise, check your configuration file, firewall and network settings.

The above steps create an Rsync server that can now be used to store backups. The example also creates a log file listing all connections. This file is stored in /var/log/rsyncd.log. This is useful if you want to debug your transfers.

To list the content of your backup target, use the following command:

rsync -avz jupiter::backup

This command lists all files present in the directory /var/backup on the server. This request is also logged in the log file /var/log/rsyncd.log. To start an actual transfer, provide a source directory. Use . for the current directory. For example, the following command copies the current directory to your Rsync backup server:

rsync -avz . jupiter::backup

By default, Rsync does not delete files and directories when it runs. To enable deletion, the additional option --delete must be stated. To ensure that no newer files are deleted, the option --update can be used instead. Any conflicts that arise must be resolved manually.

9.6 For More Information Edit source

CSync

Bidirectional file synchronizer, see https://www.csync.org/.

RSnapshot

Creates incremental backups, see http://rsnapshot.org.

Unison

A file synchronizer similar to CSync but with a graphical interface, see http://www.seas.upenn.edu/~bcpierce/unison/.

Rear

A disaster recovery framework, see the Administration Guide of the SUSE Linux Enterprise High Availability Extension https://documentation.suse.com/sle-ha-12/.

Print this page