Btrfs is a copy-on-write (COW) file system developed by Chris Mason. It is based on COW-friendly B-trees developed by Ohad Rodeh. Btrfs is a logging-style file system. Instead of journaling the block changes, it writes them in a new location, then links the change in. Until the last write, the new changes are not committed.

1.2.2 The root file system setup on SUSE Linux Enterprise Server #

  

By default, SUSE Linux Enterprise Server is set up using Btrfs and snapshots for the root partition. Snapshots allow you to easily roll back your system if needed after applying updates, or to back up files. Snapshots can easily be managed with the SUSE Snapper infrastructure as explained in Book “Administration Guide”, Chapter 7 “System recovery and snapshot management with Snapper”. For general information about the SUSE Snapper project, see the Snapper Portal wiki at OpenSUSE.org (http://snapper.io).

When using a snapshot to roll back the system, it must be ensured that data such as user's home directories, Web and FTP server contents or log files do not get lost or overwritten during a roll back. This is achieved by using Btrfs subvolumes on the root file system. Subvolumes can be excluded from snapshots. The default root file system setup on SUSE Linux Enterprise Server as proposed by YaST during the installation contains the following subvolumes. They are excluded from snapshots for the reasons given below.

/boot/grub2/i386-pc, /boot/grub2/x86_64-efi, /boot/grub2/powerpc-ieee1275, /boot/grub2/s390x-emu

A rollback of the boot loader configuration is not supported. The directories listed above are architecture-specific. The first two directories are present on AMD64/Intel 64 machines, the latter two on IBM POWER and on IBM Z, respectively.

/home

If /home does not reside on a separate partition, it is excluded to avoid data loss on rollbacks.

/opt

Third-party products usually get installed to /opt. It is excluded to avoid uninstalling these applications on rollbacks.

/srv

Contains data for Web and FTP servers. It is excluded to avoid data loss on rollbacks.

/tmp

All directories containing temporary files and caches are excluded from snapshots.

/usr/local

This directory is used when manually installing software. It is excluded to avoid uninstalling these installations on rollbacks.

/var

This directory contains many variable files, including logs, temporary caches, third party products in /var/opt, and is the default location for virtual machine images and databases. Therefore this subvolume is created to exclude all of this variable data from snapshots and has Copy-On-Write disabled.

Warning: Support for rollbacks

Rollbacks are only supported by SUSE if you do not remove any of the preconfigured subvolumes. You may, however, add subvolumes using the YaST Partitioner.

1.2.2.1 Mounting compressed Btrfs file systems #

  

Note: GRUB 2 and compressed root

GRUB 2 cannot boot from lzo or zstd compressed root file systems. Use zlib compression, or create a separate /boot partition if you wish to use lzo or zstd compression for root.

The Btrfs file system supports transparent compression. While enabled, Btrfs compresses file data when written and uncompresses file data when read.

Use the compress or compress-force mount option and select the compression algorithm, zstd, lzo, or zlib (the default). zlib compression has a higher compression ratio while lzo is faster and takes less CPU load. The zstd algorithm offers a modern compromise, with performance close to lzo and compression ratios similar to zlib.

For example:

# mount -o compress=zstd /dev/sdx /mnt

In case you create a file, write to it, and the compressed result is greater or equal to the uncompressed size, Btrfs will skip compression for future write operations forever for this file. If you do not like this behavior, use the compress-force option. This can be useful for files that have some initial non-compressible data.

Note, compression takes effect for new files only. Files that were written without compression are not compressed when the file system is mounted with the compress or compress-force option. Furthermore, files with the nodatacow attribute never get their extents compressed:

# chattr +C FILE
# mount -o nodatacow  /dev/sdx /mnt

In regard to encryption, this is independent from any compression. After you have written some data to this partition, print the details:

# btrfs filesystem show /mnt
btrfs filesystem show /mnt
Label: 'Test-Btrfs'  uuid: 62f0c378-e93e-4aa1-9532-93c6b780749d
        Total devices 1 FS bytes used 3.22MiB
      devid    1 size 2.00GiB used 240.62MiB path /dev/sdb1

If you want this to be permanent, add the compress or compress-force option into the /etc/fstab configuration file. For example:

UUID=1a2b3c4d /home btrfs subvol=@/home,compress 0 0

1.2.2.2 Mounting subvolumes #

  

A system rollback from a snapshot on SUSE Linux Enterprise Server is performed by booting from the snapshot first. This allows you to check the snapshot while running before doing the rollback. Being able to boot from snapshots is achieved by mounting the subvolumes (which would normally not be necessary).

In addition to the subvolumes listed in Section 1.2.2, “The root file system setup on SUSE Linux Enterprise Server” a volume named @ exists. This is the default subvolume that will be mounted as the root partition (/). The other subvolumes will be mounted into this volume.

When booting from a snapshot, not the @ subvolume will be used, but rather the snapshot. The parts of the file system included in the snapshot will be mounted read-only as /. The other subvolumes will be mounted writable into the snapshot. This state is temporary by default: the previous configuration will be restored with the next reboot. To make it permanent, execute the snapper rollback command. This will make the snapshot that is currently booted the new default subvolume, which will be used after a reboot.

1.2.2.3 Checking for free space #

  

File system usage is usually checked by running the df command. On a Btrfs file system, the output of df can be misleading, because in addition to the space the raw data allocates, a Btrfs file system also allocates and uses space for metadata.

Consequently a Btrfs file system may report being out of space even though it seems that plenty of space is still available. In that case, all space allocated for the metadata is used up. Use the following commands to check for used and available space on a Btrfs file system:

btrfs filesystem show

> sudo btrfs filesystem show /
Label: 'ROOT'  uuid: 52011c5e-5711-42d8-8c50-718a005ec4b3
        Total devices 1 FS bytes used 10.02GiB
        devid    1 size 20.02GiB used 13.78GiB path /dev/sda3

Shows the total size of the file system and its usage. If these two values in the last line match, all space on the file system has been allocated.

btrfs filesystem df

> sudo btrfs filesystem df /
Data, single: total=13.00GiB, used=9.61GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=768.00MiB, used=421.36MiB
GlobalReserve, single: total=144.00MiB, used=0.00B

Shows values for allocated (total) and used space of the file system. If the values for total and used for the metadata are almost equal, all space for metadata has been allocated.

btrfs filesystem usage

> sudo btrfs filesystem usage /
Overall:
    Device size:                  20.02GiB
    Device allocated:             13.78GiB
    Device unallocated:            6.24GiB
    Device missing:                  0.00B
    Used:                         10.02GiB
    Free (estimated):              9.63GiB      (min: 9.63GiB)
    Data ratio:                       1.00
    Metadata ratio:                   1.00
    Global reserve:              144.00MiB      (used: 0.00B)

             Data     Metadata  System
Id Path      single   single    single   Unallocated
-- --------- -------- --------- -------- -----------
 1 /dev/sda3 13.00GiB 768.00MiB 32.00MiB     6.24GiB
-- --------- -------- --------- -------- -----------
   Total     13.00GiB 768.00MiB 32.00MiB     6.24GiB
   Used       9.61GiB 421.36MiB 16.00KiB

Shows data similar to that of the two previous commands combined.

For more information refer to man 8 btrfs-filesystem and https://btrfs.wiki.kernel.org/index.php/FAQ.

1.2.3 Migration from ReiserFS and ext file systems to Btrfs #

  

You can migrate data volumes from existing ReiserFS or Ext (Ext2, Ext3, or Ext4) to the Btrfs file system using the btrfs-convert tool. This allows you to do an in-place conversion of unmounted (offline) file systems, which may require a bootable install media with the btrfs-convert tool. The tool constructs a Btrfs file system within the free space of the original file system, directly linking to the data contained in it. There must be enough free space on the device to create the metadata or the conversion will fail. The original file system will be intact and no free space will be occupied by the Btrfs file system. The amount of space required is dependent on the content of the file system and can vary based on the number of file system objects (such as files, directories, extended attributes) contained in it. Since the data is directly referenced, the amount of data on the file system does not impact the space required for conversion, except for files that use tail packing and are larger than about 2 KiB in size.

Warning: Root file system conversion not supported

Converting the root file system to Btrfs is not supported and not recommended. Automating such a conversion is not possible due to various steps that need to be tailored to your specific setup—the process requires a complex configuration to provide a correct rollback, /boot must be on the root file system, and the system must have specific subvolumes, etc. Either keep the existing file system or re-install the whole system from scratch.

To convert the original file system to the Btrfs file system, run:

# btrfs-convert /path/to/device

Important: Check /etc/fstab

After the conversion, you need to ensure that any references to the original file system in /etc/fstab have been adjusted to indicate that the device contains a Btrfs file system.

When converted, the contents of the Btrfs file system will reflect the contents of the source file system. The source file system will be preserved until you remove the related read-only image created at fs_root/reiserfs_saved/image. The image file is effectively a 'snapshot' of the ReiserFS file system prior to conversion and will not be modified as the Btrfs file system is modified. To remove the image file, remove the reiserfs_saved subvolume:

# btrfs subvolume delete fs_root/reiserfs_saved

To revert the file system back to the original one, use the following command:

# btrfs-convert -r /path/to/device

Warning: Lost changes

Any changes you made to the file system while it was mounted as a Btrfs file system will be lost. A balance operation must not have been performed in the interim, or the file system will not be restored correctly.

1.2.5 Btrfs quota support for subvolumes #

  

The Btrfs root file system subvolumes (for example, /var/log, /var/crash, or /var/cache) can use all the available disk space during normal operation, and cause a system malfunction. To help avoid this situation, SUSE Linux Enterprise Server offers quota support for Btrfs subvolumes. If you set up the root file system from a YaST proposal, you are ready to enable and set subvolume quotas.

1.2.5.1 Setting Btrfs quotas using YaST #

  

To set a quota for a subvolume of the root file system by using YaST, proceed as follows:

1. Start YaST and select System › Partitioner and confirm the warning with Yes.

2. In the left pane, click Btrfs.

3. In the main window, select the device for which you want to enable subvolume quotas and click Edit at the bottom.

4. In the Edit Btrfs window, activate the Enable Subvolume Quotas check box and confirm with Next.

   

   Figure 1.1: Enabling Btrfs quotas #

     

5. From the list of existing subvolumes, click the subvolume whose size you intend to limit by quota and click Edit at the bottom.

6. In the Edit subvolume of Btrfs window, activate Limit size and specify the maximum referenced size. Confirm with Accept.

   

   Figure 1.2: Setting quota for a subvolume #

     

   The new size limit will be displayed next to the subvolume name:

   

   Figure 1.3: List of subvolumes for a device #

     

7. Apply changes with Next.

1.2.5.2 Setting Btrfs quotas on the command line #

  

To set a quota for a subvolume of the root file system on the command line, proceed as follows:

1. Enable quota support:

   > sudo btrfs quota enable /

2. Get a list of subvolumes:

   > sudo btrfs subvolume list /

   Quotas can only be set for existing subvolumes.

3. Set a quota for one of the subvolumes that was listed in the previous step. A subvolume can either be identified by path (for example /var/tmp) or by 0/SUBVOLUME ID (for example 0/272). The following example sets a quota of 5 GB for /var/tmp.

   > sudo btrfs qgroup limit 5G /var/tmp

   The size can either be specified in bytes (5000000000), kilobytes (5000000K), megabytes (5000M), or gigabytes (5G). The resulting values in bytes slightly differ, since 1024 Bytes = 1 KiB, 1024 KiB = 1 MiB, etc.

4. To list the existing quotas, use the following command. The column max_rfer shows the quota in bytes.

   > sudo btrfs qgroup show -r /

Tip: Nullifying a quota

In case you want to nullify an existing quota, set a quota size of none:

> sudo btrfs qgroup limit none /var/tmp

To disable quota support for a partition and all its subvolumes, use btrfs quota disable:

> sudo btrfs quota disable /

1.2.5.3 More information #

  

See the man 8 btrfs-qgroup and man 8 btrfs-quota for more details. The UseCases page on the Btrfs wiki (https://btrfs.wiki.kernel.org/index.php/UseCases) also provides more information.

1.2.6 Swapping on Btrfs #

  

Important: Snapshots with swapping

You will not be able to create a snapshot if the source subvolume contains any enabled swap files.

SLES supports swapping to a file on the Btrfs file system if the following criteria relating to the resulting swap file are fulfilled:

• The swap file must have the NODATACOW and NODATASUM mount options.

• The swap file can not be compressed—you can ensure this by setting the NODATACOW and NODATASUM mount options. Both options disable swap file compression.

• The swap file cannot be activated while exclusive operations are running—such as device resizing, adding, removing, or replacing, or when a balancing operation is running.

• The swap file cannot be sparse.

• The swap file can not be an inline file.

• The swap file must be on a single allocation profile file system.

1.2.7 Btrfs send/receive #

  

Btrfs allows to make snapshots to capture the state of the file system. Snapper, for example, uses this feature to create snapshots before and after system changes, allowing a rollback. However, together with the send/receive feature, snapshots can also be used to create and maintain copies of a file system in a remote location. This feature can, for example, be used to do incremental backups.

A btrfs send operation calculates the difference between two read-only snapshots from the same subvolume and sends it to a file or to STDOUT. A btrfs receive operation takes the result of the send command and applies it to a snapshot.

1.2.7.1 Prerequisites #

  

To use the send/receive feature, the following requirements need to be met:

• A Btrfs file system is required on the source side (send) and on the target side (receive).

• Btrfs send/receive operates on snapshots, therefore the respective data needs to reside in a Btrfs subvolume.

• Snapshots on the source side need to be read-only.

• SUSE Linux Enterprise 12 SP2 or better. Earlier versions of SUSE Linux Enterprise do not support send/receive.

1.2.7.2 Incremental backups #

  

The following procedure shows the basic usage of Btrfs send/receive using the example of creating incremental backups of /data (source side) in /backup/data (target side). /data needs to be a subvolume.

Procedure 1.1: Initial setup #

  

1. Create the initial snapshot (called snapshot_0 in this example) on the source side and make sure it is written to the disk:

   > sudo btrfs subvolume snapshot -r /data /data/bkp_data
   sync

   A new subvolume /data/bkp_data is created. It will be used as the basis for the next incremental backup and should be kept as a reference.

2. Send the initial snapshot to the target side. Since this is the initial send/receive operation, the complete snapshot needs to be sent:

   > sudo bash -c 'btrfs send /data/bkp_data | btrfs receive /backup'

   A new subvolume /backup/bkp_data is created on the target side.

When the initial setup has been finished, you can create incremental backups and send the differences between the current and previous snapshots to the target side. The procedure is always the same:

1. Create a new snapshot on the source side.

2. Send the differences to the target side.

3. Optional: Rename and/or clean up snapshots on both sides.

Procedure 1.2: Performing an incremental backup #

  

1. Create a new snapshot on the source side and make sure it is written to the disk. In the following example the snapshot is named bkp_data_CURRENT_DATE:

   > sudo btrfs subvolume snapshot -r /data /data/bkp_data_$(date +%F)
   sync

   A new subvolume, for example /data/bkp_data_2016-07-07, is created.

2. Send the difference between the previous snapshot and the one you have created to the target side. This is achieved by specifying the previous snapshot with the option -p SNAPSHOT.

   > sudo bash -c 'btrfs send -p /data/bkp_data /data/bkp_data_2016-07-07 \
   | btrfs receive /backup'

   A new subvolume /backup/bkp_data_2016-07-07 is created.

3. As a result four snapshots, two on each side, exist:

   /data/bkp_data

   /data/bkp_data_2016-07-07

   /backup/bkp_data

   /backup/bkp_data_2016-07-07

   Now you have three options for how to proceed:

   • Keep all snapshots on both sides. With this option you can roll back to any snapshot on both sides while having all data duplicated at the same time. No further action is required. When doing the next incremental backup, keep in mind to use the next-to-last snapshot as parent for the send operation.

   • Only keep the last snapshot on the source side and all snapshots on the target side. Also allows to roll back to any snapshot on both sides—to do a rollback to a specific snapshot on the source side, perform a send/receive operation of a complete snapshot from the target side to the source side. Do a delete/move operation on the source side.

   • Only keep the last snapshot on both sides. This way you have a backup on the target side that represents the state of the last snapshot made on the source side. It is not possible to roll back to other snapshots. Do a delete/move operation on the source and the target side.

   1. To only keep the last snapshot on the source side, perform the following commands:

      > sudo btrfs subvolume delete /data/bkp_data
      > sudo mv /data/bkp_data_2016-07-07 /data/bkp_data

      The first command will delete the previous snapshot, the second command renames the current snapshot to /data/bkp_data. This ensures that the last snapshot that was backed up is always named /data/bkp_data. As a consequence, you can also always use this subvolume name as a parent for the incremental send operation.

   2. To only keep the last snapshot on the target side, perform the following commands:

      > sudo btrfs subvolume delete /backup/bkp_data
      > sudo mv /backup/bkp_data_2016-07-07 /backup/bkp_data

      The first command will delete the previous backup snapshot, the second command renames the current backup snapshot to /backup/bkp_data. This ensures that the latest backup snapshot is always named /backup/bkp_data.

Tip: Sending to a remote target side

To send the snapshots to a remote machine, use SSH:

> btrfs send /data/bkp_data | ssh root@jupiter.example.com 'btrfs receive /backup'

1.2.8 Data deduplication support #

  

Btrfs supports data deduplication by replacing identical blocks in the file system with logical links to a single copy of the block in a common storage location. SUSE Linux Enterprise Server provides the tool duperemove for scanning the file system for identical blocks. When used on a Btrfs file system, it can also be used to deduplicate these blocks and thus save space on the file system. duperemove is not installed by default. To make it available, install the package duperemove .

Note: Deduplicating large datasets

If you intend to deduplicate a large amount of files, use the --hashfile option:

> sudo duperemove --hashfile HASH_FILE file1 file2 file3

The --hashfile option stores hashes of all specified files into the HASH_FILE instead of RAM and prevents it from being exhausted. HASH_FILE is reusable—you can deduplicate changes to large datasets very quickly after an initial run that generated a baseline hash file.

duperemove can either operate on a list of files or recursively scan a directory:

> sudo duperemove OPTIONS file1 file2 file3
> sudo duperemove -r OPTIONS directory

It operates in two modes: read-only and de-duping. When run in read-only mode (that is without the -d switch), it scans the given files or directories for duplicated blocks and prints them. This works on any file system.

Running duperemove in de-duping mode is only supported on Btrfs file systems. After having scanned the given files or directories, the duplicated blocks will be submitted for deduplication.

For more information see man 8 duperemove.

Originally intended as the file system for their IRIX OS, SGI started XFS development in the early 1990s. The idea behind XFS was to create a high-performance 64-bit journaling file system to meet extreme computing challenges. XFS is very good at manipulating large files and performs well on high-end hardware. XFS is the default file system for data partitions in SUSE Linux Enterprise Server.

A quick review of XFS’s key features explains why it might prove to be a strong competitor for other journaling file systems in high-end computing.

1.3.1 XFS formats #

  

SUSE Linux Enterprise Server supports the “on-disk format” (v5) of the XFS file system. The main advantages of this format are automatic checksums of all XFS metadata, file type support, and support for a larger number of access control lists for a file.

Note that this format is not supported by SUSE Linux Enterprise kernels older than version 3.12, by xfsprogs older than version 3.2.0, and GRUB 2 versions released before SUSE Linux Enterprise 12.

Important: The V4 is deprecated

XFS is deprecating file systems with the V4 format. This file system format was created by the command:

mkfs.xfs -m crc=0 DEVICE

The format was used in SLE 11 and older releases and currently it creates a warning message by dmesg:

Deprecated V4 format (crc=0) will not be supported after September 2030

If you see the message above in the output of the dmesg command, it is recommended that you update your file system to the V5 format:

1. Back up your data to another device.

2. Create the file system on the device.

   mkfs.xfs -m crc=1 DEVICE

3. Restore the data from the backup on the updated device.

The origins of Ext2 go back to the early days of Linux history. Its predecessor, the Extended File System, was implemented in April 1992 and integrated in Linux 0.96c. The Extended File System underwent several modifications and, as Ext2, became the most popular Linux file system for years. With the creation of journaling file systems and their short recovery times, Ext2 became less important.

A brief summary of Ext2’s strengths might help understand why it was—and in some areas still is—the favorite Linux file system of many Linux users.

Ext3 was designed by Stephen Tweedie. Unlike all other next-generation file systems, Ext3 does not follow a completely new design principle. It is based on Ext2. These two file systems are very closely related to each other. An Ext3 file system can be easily built on top of an Ext2 file system. The most important difference between Ext2 and Ext3 is that Ext3 supports journaling. In summary, Ext3 has three major advantages to offer:

In 2006, Ext4 started as a fork from Ext3. It is the latest file system in the extended file system version. Ext4 was originally designed to increase storage size by supporting volumes with a size of up to 1 exbibyte, files with a size of up to 16 tebibytes and an unlimited number of subdirectories. Ext4 uses extents, instead of the traditional direct and indirect block pointers, to map the file contents. Usage of extents improves both storage and retrieval of data from disks.

Ext4 also introduces several performance enhancements such as delayed block allocation and a much faster file system checking routine. Ext4 is also more reliable by supporting journal checksums and by providing time stamps measured in nanoseconds. Ext4 is fully backward compatible to Ext2 and Ext3—both file systems can be mounted as Ext4.

Note: Ext3 funcionality on Ext4

The Ext3 funcionality is fully supported by the Ext4 driver in the Ext4 kernel module.

1.6.2 Ext4 file system inode size and number of inodes #

  

An inode stores information about the file and its block location in the file system. To allow space in the inode for extended attributes and ACLs, the default inode size was increased to 256 bytes.

When you create a new Ext4 file system, the space in the inode table is preallocated for the total number of inodes that can be created. The bytes-per-inode ratio and the size of the file system determine how many inodes are possible. When the file system is made, an inode is created for every bytes-per-inode bytes of space:

number of inodes = total size of the file system divided by the number of bytes per inode

The number of inodes controls the number of files you can have in the file system: one inode for each file.

Important: Changing the inode size of an existing ext4 file system not possible

After the inodes are allocated, you cannot change the settings for the inode size or bytes-per-inode ratio. No new inodes are possible without re-creating the file system with different settings, or unless the file system gets extended. When you exceed the maximum number of inodes, no new files can be created on the file system until some files are deleted.

When you make a new Ext4 file system, you can specify the inode size and bytes-per-inode ratio to control inode space usage and the number of files possible on the file system. If the blocks size, inode size, and bytes-per-inode ratio values are not specified, the default values in the /etc/mked2fs.conf file are applied. For information, see the mke2fs.conf(5) man page.

Use the following guidelines:

• Inode size: The default inode size is 256 bytes. Specify a value in bytes that is a power of 2 and equal to 128 or larger in bytes and up to the block size, such as 128, 256, 512, and so on. Use 128 bytes only if you do not use extended attributes or ACLs on your Ext4 file systems.

• Bytes-per-inode ratio: The default bytes-per-inode ratio is 16384 bytes. Valid bytes-per-inode ratio values must be a power of 2 equal to 1024 or greater in bytes, such as 1024, 2048, 4096, 8192, 16384, 32768, and so on. This value should not be smaller than the block size of the file system, because the block size is the smallest chunk of space used to store data. The default block size for the Ext4 file system is 4 KiB.

  In addition, consider the number of files and the size of files you need to store. For example, if your file system will have many small files, you can specify a smaller bytes-per-inode ratio, which increases the number of inodes. If your file system will have very large files, you can specify a larger bytes-per-inode ratio, which reduces the number of possible inodes.

  Generally, it is better to have too many inodes than to run out of them. If you have too few inodes and very small files, you could reach the maximum number of files on a disk that is practically empty. If you have too many inodes and very large files, you might have free space reported but be unable to use it because you cannot create new files in space reserved for inodes.

Use any of the following methods to set the inode size and bytes-per-inode ratio:

• Modifying the default settings for all new ext4 file systems: In a text editor, modify the defaults section of the /etc/mke2fs.conf file to set the inode_size and inode_ratio to the desired default values. The values apply to all new Ext4 file systems. For example:

  blocksize = 4096
  inode_size = 128
  inode_ratio = 8192

• At the command line: Pass the inode size (-I 128) and the bytes-per-inode ratio (-i 8192) to the mkfs.ext4(8) command or the mke2fs(8) command when you create a new ext4 file system. For example, use either of the following commands:

  > sudo mkfs.ext4 -b 4096 -i 8092 -I 128 /dev/sda2
  > sudo mke2fs -t ext4 -b 4096 -i 8192 -I 128 /dev/sda2

• During installation with YaST: Pass the inode size and bytes-per-inode ratio values when you create a new Ext4 file system during the installation. In the Expert Partitioner, select the partition, click Edit. Under Formatting Options, select Format deviceExt4, then click Options. In the Format options dialog, select the desired values from the Block Size in Bytes, Bytes-per-inode, and Inode Size drop-down box.

  For example, select 4096 for the Block Size in Bytes drop-down box, select 8192 from the Bytes per inode drop-down box, select 128 from the Inode Size drop-down box, then click OK.

  

1.6.3 Upgrading to Ext4 #

  

Important: Backup of data

Back up all data on the file system before performing any update of your file system.

Procedure 1.3: Upgrading to ext4 #

  

1. To upgrade from Ext2 or Ext3, you must enable the following:

   Features required by Ext4 #

     

   extents

   contiguous blocks on the hard disk that are used to keep files close together and prevent fragmentation

   unint_bg

   lazy inode table initialization

   dir_index

   hashed b-tree lookups for large directores

   on Ext2: as_journal

   enable journalling on your Ext2 file system.

   To enable the features, run:

   • on Ext3:

     # tune2fs -O extents,uninit_bg,dir_index DEVICE_NAME

   • on Ext2:

     # tune2fs -O extents,uninit_bg,dir_index,has_journal DEVICE_NAME

2. As root edit the /etc/fstab file: change the ext3 or ext2 record to ext4. The change takes effect after the next reboot.

3. To boot a file system that is setup on an ext4 parition, add the modules: ext4 and jbd in the initramfs. Open or create /etc/dracut.conf.d/filesystem.conf and add the following line:

   force_drivers+=" ext4 jbd"

   You need to overwrite the existing dracut initramfs by running:

   dracut -f

4. Reboot your system.

ReiserFS support was completely removed with SUSE Linux Enterprise Server 15. To migrate your existing partitions to Btrfs, refer to Section 1.2.3, “Migration from ReiserFS and ext file systems to Btrfs”.

Table 1.1, “File system types in Linux” summarizes some other file systems supported by Linux. They are supported mainly to ensure compatibility and interchange of data with different kinds of media or foreign operating systems.

Due to security reasons, some file systems have been blacklisted. These file systems are usually not maintained anymore and are not in common use. However, the kernel module for a blacklisted file system can be loaded, because the in-kernel API is still compatible. A combination of user-mountable file systems and the automatic mounting of file systems on removable devices could result in the situation where unprivileged users might trigger the automatic loading of kernel modules, and the removable devices could store potentially malicious data.

To get a list of currently blacklisted file systems, run the following command:

> sudo rpm -ql suse-module-tools  | sed -nE 's/.*blacklist_fs-(.*)\.conf/\1/p'

If you try to mount a device with a blacklisted file system using the mountcommand, the command outputs an error message, for example:

mount: /mnt/mx: unknown filesystem type 'minix' (hint: possibly blacklisted, see mount(8)).

To enable the mounting of the file system, you need to remove the particular file system from the blacklist. Each blacklisted file system has its own configuration file, for example, for efs it is /etc/modules.d/60-blacklist_fs-efs.conf. To remove the file system from the blacklist, you have the following options:

Even though a file system is blacklisted, you can load the corresponding kernel module for the file system directly using modprobe:

> sudo modprobe FILESYSTEM

For example, for the cramfs file system, the output looks as follows:

unblacklist: loading cramfs file system module
unblacklist: Do you want to un-blacklist cramfs permanently (<y>es/<n>o/n<e>ver)? y
unblacklist: cramfs un-blacklisted by creating /etc/modprobe.d/60-blacklist_fs-cramfs.conf

If you select yes, the modprobe command calls a script that creates a symbolic link from the configuration file of the provided file system to /dev/null. Thus, the file system is removed from the blacklist.

Originally, Linux supported a maximum file size of 2 GiB (2³¹ bytes). Unless a file system comes with large file support, the maximum file size on a 32-bit system is 2 GiB.

Currently, all our standard file systems have LFS (large file support), which gives a maximum file size of 2⁶³ bytes in theory. Table 1.2, “Maximum sizes of files and file systems (on-disk format, 4 KiB block size)” offers an overview of the current on-disk format limitations of Linux files and file systems. The numbers in the table assume that the file systems are using 4 KiB block size, which is a common standard. When using different block sizes, the results are different. The maximum file sizes in Table 1.2, “Maximum sizes of files and file systems (on-disk format, 4 KiB block size)” can be larger than the file system's actual size when using sparse blocks.

Note: Binary multiples

In this document: 1024 Bytes = 1 KiB; 1024 KiB = 1 MiB; 1024 MiB = 1 GiB; 1024 GiB = 1 TiB; 1024 TiB = 1 PiB; 1024 PiB = 1 EiB (see also NIST: Prefixes for Binary Multiples.

Important: Limitations

Table 1.2, “Maximum sizes of files and file systems (on-disk format, 4 KiB block size)” describes the limitations regarding the on-disk format. The Linux kernel imposes its own limits on the size of files and file systems handled by it. These are as follows:

File size

On 32-bit systems, files cannot exceed 2 TiB (2⁴¹ bytes).

File system size

File systems can be up to 2⁷³ bytes in size. However, this limit is still out of reach for the currently available hardware.

Table 1.3, “Storage limitations” summarizes the kernel limits for storage associated with SUSE Linux Enterprise Server.

On solid-state drives (SSDs) and thinly provisioned volumes, it is useful to trim blocks that are not in use by the file system. SUSE Linux Enterprise Server fully supports unmap and TRIM operations on all file systems supporting them.

There are two types of commonly used TRIM—online TRIM and periodic TRIM. The most suitable way of trimming devices depends on your use case. In general, it is recommended to use periodic TRIM, especially if the device has enough free blocks. If the device is often near its full capacity, online TRIM is preferable.

Important: TRIM support on devices

Always verify that your device supports the TRIM operation before you attempt to use it. Otherwise, you might lose your data on that device. To verify the TRIM support, run the command:

> sudo lsblk --discard

The command outputs information about all available block devices. If the values of the columns DISC-GRAN and DISC-MAX are non-zero, the device supports the TRIM operation.

> sudo systemctl enable fstrim.timer

1.12.2 Online TRIM #

  

Online TRIM of a device is performed each time data is written to the device.

To enable online TRIM on a device, add the discard option to the /etc/fstab file as follows:

UID=83df497d-bd6d-48a3-9275-37c0e3c8dc74  /  btrfs  defaults,discard

Alternatively, on the Ext4 file system you can use the tune2fs command to set the discard option in /etc/fstab:

> sudo tune2fs -o discard DEVICE

The discard option is also added to /etc/fstab in case the device was mounted by mount with the discard option:

> sudo mount -o discard DEVICE

Note: Drawbacks of online TRIM

Using the discard option may decrease life time of some lower-quality SSD devices. Online TRIM can also impact the performance of the device, for example, if a larger amount of data is deleted. In this situation, an erase block might be reallocated, and shortly afterwards, the same erase block might be marked as unused again.

This section describes some known issues and possible solutions for file systems.

1.13.2 Btrfs: balancing data across devices #

  

The btrfs balance command is part of the btrfs-progs package. It balances block groups on Btrfs file systems in the following example situations:

• Assume you have a 1 TB drive with 600 GB used by data and you add another 1 TB drive. The balancing will theoretically result in having 300 GB of used space on each drive.

• You have a lot of near-empty chunks on a device. Their space will not be available until the balancing has cleared those chunks.

• You need to compact half-empty block group based on the percentage of their usage. The following command will balance block groups whose usage is 5% or less:

  > sudo btrfs balance start -dusage=5 /

  

  Tip

  The /usr/lib/systemd/system/btrfs-balance.timer timer takes care of cleaning up unused block groups on a monthly basis.

• You need to clear out non-full portions of block devices and spread data more evenly.

• You need to migrate data between different RAID types. For example, to convert data on a set of disks from RAID1 to RAID5, run the following command:

  > sudo btrfs balance start -dprofiles=raid1,convert=raid5 /

Tip

To fine-tune the default behavior of balancing data on Btrfs file systems—for example, how frequently or which mount points to balance— inspect and customize /etc/sysconfig/btrfsmaintenance. The relevant options start with BTRFS_BALANCE_.

For details about the btrfs balance command usage, see its manual pages (man 8 btrfs-balance).

Each of the file system projects described above maintains its own home page on which to find mailing list information, further documentation, and FAQs:

An in-depth comparison of file systems (not only Linux file systems) is available from the Wikipedia project in Comparison of File Systems (http://en.wikipedia.org/wiki/Comparison_of_file_systems#Comparison).

It is strongly recommended to use the YaST Partitioner to resize partitions or logical volumes. When doing so, the file system will automatically be adjusted to the new size of the partition or volume. However, there are some cases where you need to resize the file system manually, because they are not supported by YaST:

Resizing any file system involves some risks that can potentially result in losing data.

Warning: Back up your data

To avoid data loss, ensure that you back up your data before you begin any resizing task.

Consider the following guidelines when planning to resize a file system.

2.2.3 Decreasing the size of a file system #

  

When decreasing the size of the file system on a device, ensure that the new size satisfies the following conditions:

• The new size must be greater than the size of the existing data; otherwise, data loss occurs.

• The new size must be equal to or less than the current device size because the file system size cannot extend beyond the space available.

If you plan to also decrease the size of the logical volume that holds the file system, ensure that you decrease the size of the file system before you attempt to decrease the size of the device or logical volume.

Important: XFS

Decreasing the size of a file system formatted with XFS is not possible, since such a feature is not supported by XFS.

The size of a Btrfs file system can be changed by using the btrfs filesystem resize command when the file system is mounted. Increasing and decreasing the size are both supported while the file system is mounted.

The size of an XFS file system can be increased by using the xfs_growfs command when the file system is mounted. Reducing the size of an XFS file system is not possible.

The size of Ext2, Ext3, and Ext4 file systems can be increased by using the resize2fs command, regardless of whether the respective partition is mounted or not. To decrease the size of an Ext file system it needs to be unmounted.

A UUID (Universally Unique Identifier) is a 128-bit number for a file system that is unique on both the local system and across other systems. It is randomly generated with system hardware information and time stamps as part of its seed. UUIDs are commonly used to uniquely tag devices.

Using non-persistent “traditional” device names such as /dev/sda1 may render the system unbootable when adding storage. For example, if root (/) is assigned to /dev/sda1, it might be reassigned to /dev/sdg1 after a SAN has been attached or additional hard disks have been applied to the system. In this case the boot loader configuration and the /etc/fstab file need to be adjusted, otherwise the system will no longer boot.

By default UUID are used in the boot loader and /etc/fstab files for the boot device. The UUID is a property of the file system and can change if you reformat the drive. Other alternatives to using UUIDs of device names would be to identify devices by ID or label.

You can also use the UUID as criterion for assembling and activating software RAID devices. When a RAID is created, the md driver generates a UUID for the device, and stores the value in the md superblock.

You can find the UUID for any block device in the /dev/disk/by-uuid directory. For example, a UUID entry looks like this:

> ls -og /dev/disk/by-uuid/
lrwxrwxrwx 1 10 Dec  5 07:48 e014e482-1c2d-4d09-84ec-61b3aefde77a -> ../../sda1

Starting with Linux kernel 2.6, udev provides a user space solution for the dynamic /dev directory, with persistent device naming. As part of the hotplug system, udev is executed if a device is added to or removed from the system.

A list of rules is used to match against specific device attributes. The udev rules infrastructure (defined in the /etc/udev/rules.d directory) provides stable names for all disk devices, regardless of their order of recognition or the connection used for the device. The udev tools examine every appropriate block device that the kernel creates to apply naming rules based on certain buses, drive types, or file systems. For information about how to define your own rules for udev, see Writing udev Rules.

Along with the dynamic kernel-provided device node name, udev maintains classes of persistent symbolic links pointing to the device in the /dev/disk directory, which is further categorized by the by-id, by-label, by-path, and by-uuid subdirectories.

Note: UUID generators

Other programs besides udev, such as LVM or md, might also generate UUIDs, but they are not listed in /dev/disk.

For more information about using udev for managing devices, see Book “Administration Guide”, Chapter 24 “Dynamic kernel device management with udev”.

For more information about udev commands, see man 7 udev.

Some types of storage devices require network to be configured and available before systemd.mount starts to mount the devices. To postpone mounting of these types of devices, add the _netdev option to the /etc/fstab file for each particular network storage device. An example follows:

mars.example.org:/nfsexport  /shared   nfs  defaults,_netdev    0  0

This section explains several terms often used when describing cache related features:

Following are the basic caching modes that multi-tier caches use: write-back, write-through, write-around and pass-through.

bcache is a Linux kernel block layer cache. It allows one or more fast disk drives (such as SSDs) to act as a cache for one or more slower hard disks. bcache supports write-through and write-back, and is independent of the file system used. By default it caches random reads and writes only, which SSDs excel at. It is suitable for desktops, servers, and high end storage arrays as well.

lvmcache is a caching mechanism consisting of logical volumes (LVs). It uses the dm-cache kernel driver and supports write-through (default) and write-back caching modes. lvmcache improves performance of a large and slow LV by dynamically migrating some of its data to a faster and smaller LV. For more information on LVM, see Part II, “Logical volumes (LVM)”.

LVM refers to the small, fast LV as a cache pool LV. The large, slow LV is called the origin LV. Because of requirements from dm-cache, LVM further splits the cache pool LV into two devices: the cache data LV and cache metadata LV. The cache data LV is where copies of data blocks are kept from the origin LV to increase speed. The cache metadata LV holds the accounting information that specifies where data blocks are stored.

> sudo lvconvert --splitcache vg/origin_lv

> sudo lvremove vg/cache_data_lv

> sudo lvconvert --uncache vg/origin_lv

> sudo lvremove vg/origin_lv

LVM enables flexible distribution of hard disk space over several physical volumes (hard disks, partitions, LUNs). It was developed because the need to change the segmentation of hard disk space might arise only after the initial partitioning has already been done during installation. Because it is difficult to modify partitions on a running system, LVM provides a virtual pool (volume group or VG) of storage space from which logical volumes (LVs) can be created as needed. The operating system accesses these LVs instead of the physical partitions. Volume groups can span more than one disk, so that several disks or parts of them can constitute one single VG. In this way, LVM provides a kind of abstraction from the physical disk space that allows its segmentation to be changed in a much easier and safer way than through physical repartitioning.

Figure 5.1, “Physical partitioning versus LVM” compares physical partitioning (left) with LVM segmentation (right). On the left side, one single disk has been divided into three physical partitions (PART), each with a mount point (MP) assigned so that the operating system can access them. On the right side, two disks have been divided into two and three physical partitions each. Two LVM volume groups (VG 1 and VG 2) have been defined. VG 1 contains two partitions from DISK 1 and one from DISK 2. VG 2 contains the remaining two partitions from DISK 2.

Figure 5.1: Physical partitioning versus LVM #

  

In LVM, the physical disk partitions that are incorporated in a volume group are called physical volumes (PVs). Within the volume groups in Figure 5.1, “Physical partitioning versus LVM”, four logical volumes (LV 1 through LV 4) have been defined, which can be used by the operating system via the associated mount points (MP). The border between different logical volumes need not be aligned with any partition border. See the border between LV 1 and LV 2 in this example.

LVM features:

Note: LVM and RAID

Even though LVM also supports RAID levels 0, 1, 4, 5 and 6, we recommend using mdraid (see Chapter 7, Software RAID configuration). However, LVM works fine with RAID 0 and 1, as RAID 0 is similar to common logical volume management (individual logical blocks are mapped onto blocks on the physical devices). LVM used on top of RAID 1 can keep track of mirror synchronization and is fully able to manage the synchronization process. With higher RAID levels you need a management daemon that monitors the states of attached disks and can inform administrators if there is a problem in the disk array. LVM includes such a daemon, but in exceptional situations such as a device failure, the daemon does not work properly.

Warning: IBM Z: LVM root file system

If you configure the system with a root file system on LVM or software RAID array, you must place /boot on a separate, non-LVM or non-RAID partition, otherwise the system will fail to boot. The recommended size for such a partition is 500 MB and the recommended file system is Ext4.

With these features, using LVM already makes sense for heavily-used home PCs or small servers. If you have a growing data stock, as in the case of databases, music archives, or user directories, LVM is especially useful. It allows file systems that are larger than the physical hard disk. However, keep in mind that working with LVM is different from working with conventional partitions.

You can manage new or existing LVM storage objects by using the YaST Partitioner. Instructions and further information about configuring LVM are available in the official LVM HOWTO.

An LVM volume group (VG) organizes the Linux LVM partitions into a logical pool of space. You can carve out logical volumes from the available space in the group. The Linux LVM partitions in a group can be on the same or different disks. You can add partitions or entire disks to expand the size of the group.

To use an entire disk, it must not contain any partitions. When using partitions, they must not be mounted. YaST will automatically change their partition type to 0x8E Linux LVM when adding them to a VG.

A logical volume provides a pool of space similar to what a hard disk does. To make this space usable, you need to define logical volumes. A logical volume is similar to a regular partition—you can format and mount it.

Use The YaST Partitioner to create logical volumes from an existing volume group. Assign at least one logical volume to each volume group. You can create new logical volumes as needed until all free space in the volume group has been exhausted. An LVM logical volume can optionally be thinly provisioned, allowing you to create logical volumes with sizes that overbook the available free space (see Section 5.3.1, “Thinly provisioned logical volumes” for more information).

5.3.1 Thinly provisioned logical volumes #

  

An LVM logical volume can optionally be thinly provisioned. Thin provisioning allows you to create logical volumes with sizes that overbook the available free space. You create a thin pool that contains unused space reserved for use with an arbitrary number of thin volumes. A thin volume is created as a sparse volume and space is allocated from a thin pool as needed. The thin pool can be expanded dynamically when needed for cost-effective allocation of storage space. Thinly provisioned volumes also support snapshots which can be managed with Snapper—see Book “Administration Guide”, Chapter 7 “System recovery and snapshot management with Snapper” for more information.

To set up a thinly provisioned logical volume, proceed as described in Procedure 5.1, “Setting up a logical volume”. When it comes to choosing the volume type, do not choose Normal Volume, but rather Thin Volume or Thin Pool.

Thin pool

The logical volume is a pool of space that is reserved for use with thin volumes. The thin volumes can allocate their needed space from it on demand.

Thin volume

The volume is created as a sparse volume. The volume allocates needed space on demand from a thin pool.

Important: Thinly provisioned volumes in a cluster

To use thinly provisioned volumes in a cluster, the thin pool and the thin volumes that use it must be managed in a single cluster resource. This allows the thin volumes and thin pool to always be mounted exclusively on the same node.

> sudo lvcreate -L 500G -m 2 -n lv1 vg1

> sudo lvcreate --type raid1 -m 1 -L 1G -n lv1 vg1

Activation behavior for non-root LVM volume groups is controlled in the /etc/lvm/lvm.conf file and by the auto_activation_volume_list parameter. By default, the parameter is empty and all volumes are activated. To activate only some volume groups, add the names in quotes and separate them with commas, for example:

auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]

If you have defined a list in the auto_activation_volume_list parameter, the following will happen:

By default, non-root LVM volume groups are automatically activated on system restart by Dracut. This parameter allows you to activate all volume groups on system restart, or to activate only specified non-root LVM volume groups.

The space provided by a volume group can be expanded at any time in the running system without service interruption by adding physical volumes. This will allow you to add logical volumes to the group or to expand the size of existing volumes as described in Section 5.6, “Resizing a logical volume”.

It is also possible to reduce the size of the volume group by removing physical volumes. YaST only allows to remove physical volumes that are currently unused. To find out which physical volumes are currently in use, run the following command. The partitions (physical volumes) listed in the PE Ranges column are the ones in use:

> sudo pvs -o vg_name,lv_name,pv_name,seg_pe_ranges
root's password:
  VG   LV    PV         PE Ranges
             /dev/sda1
  DATA DEVEL /dev/sda5  /dev/sda5:0-3839
  DATA       /dev/sda5
  DATA LOCAL /dev/sda6  /dev/sda6:0-2559
  DATA       /dev/sda7
  DATA       /dev/sdb1
  DATA       /dev/sdc1

In case there is unused free space available in the volume group, you can enlarge a logical volume to provide more usable space. You may also reduce the size of a volume to free space in the volume group that can be used by other logical volumes.

Note: “Online” resizing

When reducing the size of a volume, YaST automatically resizes its file system, too. Whether a volume that is currently mounted can be resized “online” (that is while being mounted), depends on its file system. Growing the file system online is supported by Btrfs, XFS, Ext3, and Ext4.

Shrinking the file system online is only supported by Btrfs. To shrink the Ext2/3/4 file systems, you need to unmount them. Shrinking volumes formatted with XFS is not possible, since XFS does not support file system shrinking.

If there is an error on the LVM storage, the scanning of LVM volumes may prevent entering the emergency/rescue shell. Thus, further problem diagnosis is not possible. To disable this scanning in case of an LVM storage failure, you can pass the nolvm option on the kernel command line.

For information about using LVM commands, see the man pages for the commands described in the following table. All commands need to be executed with root privileges. Either use sudo COMMAND (recommended), or execute them directly as root.

Tip: Bypassing udev on volume creation

In case you want to manage LV device nodes and symbolic links by using LVM instead of by using udev rules, you can achieve this by disabling notifications from udev with one of the following methods:

• Configure activation/udev_rules = 0 and activation/udev_sync = 0 in /etc/lvm/lvm.conf.

  Note that specifying --nodevsync with the lvcreate command has the same effect as activation/udev_sync = 0; setting activation/udev_rules = 0 is still required.

• Setting the environment variable DM_DISABLE_UDEV:

  export DM_DISABLE_UDEV=1

  This will also disable notifications from udev. In addition, all udev related settings from /etc/lvm/lvm.conf will be ignored.

5.9.1 Resizing a logical volume with commands #

  

The lvresize, lvextend, and lvreduce commands are used to resize logical volumes. See the man pages for each of these commands for syntax and options information. To extend an LV there must be enough unallocated space available on the VG.

The recommended way to grow or shrink a logical volume is to use the YaST Partitioner. When using YaST, the size of the file system in the volume will automatically be adjusted, too.

LVs can be extended or shrunk manually while they are being used, but this may not be true for a file system on them. Extending or shrinking the LV does not automatically modify the size of file systems in the volume. You must use a different command to grow the file system afterward. For information about resizing file systems, see Chapter 2, Resizing file systems.

Ensure that you use the right sequence when manually resizing an LV:

• If you extend an LV, you must extend the LV before you attempt to grow the file system.

• If you shrink an LV, you must shrink the file system before you attempt to shrink the LV.

To extend the size of a logical volume:

1. Open a terminal.

2. If the logical volume contains an Ext2 or Ext4 file system, which do not support online growing, dismount it. In case it contains file systems that are hosted for a virtual machine (such as a Xen VM), shut down the VM first.

3. At the terminal prompt, enter the following command to grow the size of the logical volume:

   > sudo lvextend -L +SIZE /dev/VG_NAME/LV_NAME

   For SIZE, specify the amount of space you want to add to the logical volume, such as 10 GB. Replace /dev/VG_NAME/LV_NAME with the Linux path to the logical volume, such as /dev/LOCAL/DATA. For example:

   > sudo lvextend -L +10GB /dev/vg1/v1

4. Adjust the size of the file system. See Chapter 2, Resizing file systems for details.

5. In case you have dismounted the file system, mount it again.

For example, to extend an LV with a (mounted and active) Btrfs on it by 10 GB:

> sudo lvextend −L +10G /dev/LOCAL/DATA
> sudo btrfs filesystem resize +10G /dev/LOCAL/DATA

To shrink the size of a logical volume:

1. Open a terminal.

2. If the logical volume does not contain a Btrfs file system, dismount it. In case it contains file systems that are hosted for a virtual machine (such as a Xen VM), shut down the VM first. Note that volumes with the XFS file system cannot be reduced in size.

3. Adjust the size of the file system. See Chapter 2, Resizing file systems for details.

4. At the terminal prompt, enter the following command to shrink the size of the logical volume to the size of the file system:

   > sudo lvreduce /dev/VG_NAME/LV_NAME

5. In case you have unmounted the file system, mount it again.

For example, to shrink an LV with a Btrfs on it by 5 GB:

> sudo btrfs filesystem resize -size 5G /dev/LOCAL/DATA
sudo lvreduce /dev/LOCAL/DATA

Tip: Resizing the Volume and the File System with a Single Command

Starting with SUSE Linux Enterprise Server 12 SP1, lvextend, lvresize, and lvreduce support the option --resizefs which will not only change the size of the volume, but will also resize the file system. Therefore the examples for lvextend and lvreduce shown above can alternatively be run as follows:

> sudo lvextend --resizefs −L +10G /dev/LOCAL/DATA
> sudo lvreduce  --resizefs -L -5G /dev/LOCAL/DATA

Note that the --resizefs is supported for the following file systems: ext2/3/4, Btrfs, XFS. Resizing Btrfs with this option is currently only available on SUSE Linux Enterprise Server, since it is not yet accepted upstream.

A tag is an unordered keyword or term assigned to the metadata of a storage object. Tagging allows you to classify collections of LVM storage objects in ways that you find useful by attaching an unordered list of tags to their metadata.

tags {
   # Enable hostname tags
   hosttags = 1
}

tags {

   tag1 { }
      # Tag does not require a match to be set.

   tag2 {
      # If no exact match, tag is not set.
      host_list = [ "hostname1", "hostname2" ]
   }
}

  activation {
      volume_list = [ "vg1/lvol0", "@database" ]
  }

A file system snapshot contains metadata about itself and data blocks from a source logical volume that has changed since the snapshot was taken. When you access data via the snapshot, you see a point-in-time copy of the source logical volume. There is no need to restore data from backup media or to overwrite the changed data.

Important: Mounting volumes with snapshots

During the snapshot’s lifetime, the snapshot must be mounted before its source logical volume can be mounted.

LVM volume snapshots allow you to create a backup from a point-in-time view of the file system. The snapshot is created instantly and persists until you delete it. You can back up the file system from the snapshot while the volume itself continues to be available for users. The snapshot initially contains some metadata about the snapshot, but no actual data from the source logical volume. Snapshot uses copy-on-write technology to detect when data changes in an original data block. It copies the value it held when the snapshot was taken to a block in the snapshot volume, then allows the new data to be stored in the source block. As more blocks change from their original value on the source logical volume, the snapshot size grows.

When you are sizing the snapshot, consider how much data is expected to change on the source logical volume and how long you plan to keep the snapshot. The amount of space that you allocate for a snapshot volume can vary, depending on the size of the source logical volume, how long you plan to keep the snapshot, and the number of data blocks that are expected to change during the snapshot’s lifetime. The snapshot volume cannot be resized after it is created. As a guide, create a snapshot volume that is about 10% of the size of the original logical volume. If you anticipate that every block in the source logical volume will change at least one time before you delete the snapshot, then the snapshot volume should be at least as large as the source logical volume plus some additional space for metadata about the snapshot volume. Less space is required if the data changes infrequently or if the expected lifetime is sufficiently brief.

In LVM2, snapshots are read/write by default. When you write data directly to the snapshot, that block is marked in the exception table as used, and never gets copied from the source logical volume. You can mount the snapshot volume, and test application changes by writing data directly to the snapshot volume. You can easily discard the changes by dismounting the snapshot, removing the snapshot, and then remounting the source logical volume.

In a virtual guest environment, you can use the snapshot function for LVM logical volumes you create on the server’s disks, as you would on a physical server.

In a virtual host environment, you can use the snapshot function to back up the virtual machine’s storage back-end, or to test changes to a virtual machine image, such as for patches or upgrades, without modifying the source logical volume. The virtual machine must be using an LVM logical volume as its storage back-end, as opposed to using a virtual disk file. You can mount the LVM logical volume and use it to store the virtual machine image as a file-backed disk, or you can assign the LVM logical volume as a physical disk to write to it as a block device.

Beginning in SLES 11 SP3, an LVM logical volume snapshot can be thinly provisioned. Thin provisioning is assumed if you create a snapshot without a specified size. The snapshot is created as a thin volume that uses space as needed from a thin pool. A thin snapshot volume has the same characteristics as any other thin volume. You can independently activate the volume, extend the volume, rename the volume, remove the volume, and even snapshot the volume.

Important: Thinly provisioned volumes in a cluster

To use thinly provisioned snapshots in a cluster, the source logical volume and its snapshots must be managed in a single cluster resource. This allows the volume and its snapshots to always be mounted exclusively on the same node.

When you are done with the snapshot, it is important to remove it from the system. A snapshot eventually fills up completely as data blocks change on the source logical volume. When the snapshot is full, it is disabled, which prevents you from remounting the source logical volume.

If you create multiple snapshots for a source logical volume, remove the snapshots in a last created, first deleted order.

The Logical Volume Manager (LVM) can be used for creating snapshots of your file system.

Open a terminal and enter

> sudo lvcreate -s [-L <size>] -n SNAP_VOLUME SOURCE_VOLUME_PATH

If no size is specified, the snapshot is created as a thin snapshot.

For example:

> sudo lvcreate -s -L 1G -n linux01-snap /dev/lvm/linux01

The snapshot is created as the /dev/lvm/linux01-snap volume.

Open a terminal and enter

> sudo lvdisplay SNAP_VOLUME

For example:

> sudo lvdisplay /dev/vg01/linux01-snap

--- Logical volume ---
  LV Name                /dev/lvm/linux01
  VG Name                vg01
  LV UUID                QHVJYh-PR3s-A4SG-s4Aa-MyWN-Ra7a-HL47KL
  LV Write Access        read/write
  LV snapshot status     active destination for /dev/lvm/linux01
  LV Status              available
  # open                 0
  LV Size                80.00 GB
  Current LE             1024
  COW-table size         8.00 GB
  COW-table LE           512
  Allocated to snapshot  30%
  Snapshot chunk size    8.00 KB
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:5

Open a terminal and enter

> sudo lvremove SNAP_VOLUME_PATH

For example:

> sudo lvremove /dev/lvmvg/linux01-snap

Using an LVM logical volume for a virtual machine’s back-end storage allows flexibility in administering the underlying device, such as making it easier to move storage objects, create snapshots, and back up data. You can mount the LVM logical volume and use it to store the virtual machine image as a file-backed disk, or you can assign the LVM logical volume as a physical disk to write to it as a block device. You can create a virtual disk image on the LVM logical volume, then snapshot the LVM.

You can leverage the read/write capability of the snapshot to create different instances of a virtual machine, where the changes are made to the snapshot for a particular virtual machine instance. You can create a virtual disk image on an LVM logical volume, snapshot the source logical volume, and modify the snapshot for a particular virtual machine instance. You can create another snapshot of the source logical volume, and modify it for a different virtual machine instance. The majority of the data for the different virtual machine instances resides with the image on the source logical volume.

You can also leverage the read/write capability of the snapshot to preserve the virtual disk image while testing patches or upgrades in the guest environment. You create a snapshot of the LVM volume that contains the image, and then run the virtual machine on the snapshot location. The source logical volume is unchanged, and all changes for that machine are written to the snapshot. To return to the source logical volume of the virtual machine image, you power off the virtual machine, then remove the snapshot from the source logical volume. To start over, you re-create the snapshot, mount the snapshot, and restart the virtual machine on the snapshot image.

The following procedure uses a file-backed virtual disk image and the Xen hypervisor. You can adapt the procedure in this section for other hypervisors that run on the SUSE Linux Enterprise platform, such as KVM. To run a file-backed virtual machine image from the snapshot volume:

Snapshots can be useful if you need to roll back or restore data on a volume to a previous state. For example, you might need to revert data changes that resulted from an administrator error or a failed or undesirable package installation or upgrade.

You can use the lvconvert --merge command to revert the changes made to an LVM logical volume. The merging begins as follows:

After a merge begins, the merge continues automatically after server restarts until it is complete. A new snapshot cannot be created for the source logical volume while a merge is in progress.

While the merge is in progress, reads or writes to the source logical volume are transparently redirected to the snapshot that is being merged. This allows users to immediately view and access the data as it was when the snapshot was created. They do not need to wait for the merge to complete.

When the merge is complete, the source logical volume contains the same data as it did when the snapshot was taken, plus any data changes made after the merge began. The resulting logical volume has the source logical volume’s name, minor number, and UUID. The source logical volume is automatically remounted, and the snapshot volume is removed.

This section describes common RAID levels 0, 1, 2, 3, 4, 5, and nested RAID levels.

The YaST soft RAID configuration can be reached from the YaST Expert Partitioner. This partitioning tool also enables you to edit and delete existing partitions and create new ones that should be used with soft RAID. These instructions apply on setting up RAID levels 0, 1, 5, and 6. Setting up RAID 10 configurations is explained in Chapter 9, Creating software RAID 10 devices.

Important: RAID on disks

While the partitioner makes it possible to create a RAID on top of disks instead of partitions, we do not recommend this approach for a number of reasons. Installing a bootloader on such RAID is not supported, so you need to use a separate device for booting. Tools like fdisk and parted do not work properly with such RAIDs, which may lead to incorrect diagnosis and actions by a person who is unaware of the RAID's particular setup.

You can run mdadm as a daemon in the monitor mode to monitor your software RAID. In the monitor mode, mdadm performs regular checks on the array for disk failures. If there is a failure, mdadm sends an email to the administrator. To define the time interval of the checks, run the following command:

mdadm --monitor --mail=root@localhost --delay=1800 /dev/md2

The command above turns on monitoring of the /dev/md2 array in intervals of 1800 seconds. In the event of a failure, an email will be sent to root@localhost.

Note: RAID checks are enabled by default

The RAID checks are enabled by default. It may happen that the interval between each check is not long enough and you may encounter warnings. Thus, you can increase the interval by setting a higher value with the delay option.

Configuration instructions and more details for soft RAID can be found in the HOWTOs at:

Linux RAID mailing lists are also available, such as linux-raid at http://marc.info/?l=linux-raid.

Ensure that your configuration meets the following requirements:

A nested RAID device consists of a RAID array that uses another RAID array as its basic element, instead of using physical disks. The goal of this configuration is to improve the performance and fault tolerance of the RAID. Setting up nested RAID levels is not supported by YaST, but can be done by using the mdadm command line tool.

Based on the order of nesting, two different nested RAIDs can be set up. This document uses the following terminology:

The following table describes the advantages and disadvantages of RAID 10 nesting as 1+0 versus 0+1. It assumes that the storage objects you use reside on different disks, each with a dedicated I/O capability.

9.1.1 Creating nested RAID 10 (1+0) with mdadm #

  

A nested RAID 1+0 is built by creating two or more RAID 1 (mirror) devices, then using them as component devices in a RAID 0.

Important: Multipathing

If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 18, Managing multipath I/O for devices.

The procedure in this section uses the device names shown in the following table. Ensure that you modify the device names with the names of your own devices.

Table 9.2: Scenario for creating a RAID 10 (1+0) by nesting #

  

Raw Devices	RAID 1 (mirror)	RAID 1+0 (striped mirrors)
/dev/sdb1 /dev/sdc1	/dev/md0	/dev/md2
/dev/sdd1 /dev/sde1	/dev/md1

1. Open a terminal.

2. If necessary, create four 0xFD Linux RAID partitions of equal size using a disk partitioner such as parted.

3. Create two software RAID 1 devices, using two different devices for each device. At the command prompt, enter these two commands:

   > sudo mdadm --create /dev/md0 --run --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
   sudo mdadm --create /dev/md1 --run --level=1 --raid-devices=2 /dev/sdd1 /dev/sde1

4. Create the nested RAID 1+0 device. At the command prompt, enter the following command using the software RAID 1 devices you created in the previous step:

   > sudo mdadm --create /dev/md2 --run --level=0 --chunk=64 \
   --raid-devices=2 /dev/md0 /dev/md1

   The default chunk size is 64 KB.

5. Create a file system on the RAID 1+0 device /dev/md2, for example an XFS file system:

   > sudo mkfs.xfs /dev/md2

   Modify the command to use a different file system.

6. Edit the /etc/mdadm.conf file or create it, if it does not exist (for example by running sudo vi /etc/mdadm.conf). Add the following lines (if the file already exists, the first line probably already exists).

   DEVICE containers partitions
   ARRAY /dev/md0 UUID=UUID
   ARRAY /dev/md1 UUID=UUID
   ARRAY /dev/md2 UUID=UUID

   The UUID of each device can be retrieved with the following command:

   > sudo mdadm -D /dev/DEVICE | grep UUID

7. Edit the /etc/fstab file to add an entry for the RAID 1+0 device /dev/md2. The following example shows an entry for a RAID device with the XFS file system and /data as a mount point.

   /dev/md2 /data xfs defaults 1 2

8. Mount the RAID device:

   > sudo mount /data

9.1.2 Creating nested RAID 10 (0+1) with mdadm #

  

A nested RAID 0+1 is built by creating two to four RAID 0 (striping) devices, then mirroring them as component devices in a RAID 1.

Important: Multipathing

If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 18, Managing multipath I/O for devices.

In this configuration, spare devices cannot be specified for the underlying RAID 0 devices because RAID 0 cannot tolerate a device loss. If a device fails on one side of the mirror, you must create a replacement RAID 0 device, than add it into the mirror.

The procedure in this section uses the device names shown in the following table. Ensure that you modify the device names with the names of your own devices.

Table 9.3: Scenario for creating a RAID 10 (0+1) by nesting #

  

Raw Devices	RAID 0 (stripe)	RAID 0+1 (mirrored stripes)
/dev/sdb1 /dev/sdc1	/dev/md0	/dev/md2
/dev/sdd1 /dev/sde1	/dev/md1

1. Open a terminal.

2. If necessary, create four 0xFD Linux RAID partitions of equal size using a disk partitioner such as parted.

3. Create two software RAID 0 devices, using two different devices for each RAID 0 device. At the command prompt, enter these two commands:

   > sudo mdadm --create /dev/md0 --run --level=0 --chunk=64 \
   --raid-devices=2 /dev/sdb1 /dev/sdc1
   sudo mdadm --create /dev/md1 --run --level=0 --chunk=64 \
   --raid-devices=2 /dev/sdd1 /dev/sde1

   The default chunk size is 64 KB.

4. Create the nested RAID 0+1 device. At the command prompt, enter the following command using the software RAID 0 devices you created in the previous step:

   > sudo mdadm --create /dev/md2 --run --level=1 --raid-devices=2 /dev/md0 /dev/md1

5. Create a file system on the RAID 1+0 device /dev/md2, for example an XFS file system:

   > sudo mkfs.xfs /dev/md2

   Modify the command to use a different file system.

6. Edit the /etc/mdadm.conf file or create it, if it does not exist (for example by running sudo vi /etc/mdadm.conf). Add the following lines (if the file exists, the first line probably already exists, too).

   DEVICE containers partitions
   ARRAY /dev/md0 UUID=UUID
   ARRAY /dev/md1 UUID=UUID
   ARRAY /dev/md2 UUID=UUID

   The UUID of each device can be retrieved with the following command:

   > sudo mdadm -D /dev/DEVICE | grep UUID

7. Edit the /etc/fstab file to add an entry for the RAID 1+0 device /dev/md2. The following example shows an entry for a RAID device with the XFS file system and /data as a mount point.

   /dev/md2 /data xfs defaults 1 2

8. Mount the RAID device:

   > sudo mount /data

YaST (and mdadm with the --level=10 option) creates a single complex software RAID 10 that combines features of both RAID 0 (striping) and RAID 1 (mirroring). Multiple copies of all data blocks are arranged on multiple drives following a striping discipline. Component devices should be the same size.

The complex RAID 10 is similar in purpose to a nested RAID 10 (1+0), but differs in the following ways:

9.2.2 Layout #

  

The complex RAID 10 setup supports three different layouts which define how the data blocks are arranged on the disks. The available layouts are near (default), far and offset. They have different performance characteristics, so it is important to choose the right layout for your workload.

9.2.2.1 Near layout #

  

With the near layout, copies of a block of data are striped near each other on different component devices. That is, multiple copies of one data block are at similar offsets in different devices. Near is the default layout for RAID 10. For example, if you use an odd number of component devices and two copies of data, some copies are perhaps one chunk further into the device.

The near layout for the complex RAID 10 yields read and write performance similar to RAID 0 over half the number of drives.

Near layout with an even number of disks and two replicas:

sda1 sdb1 sdc1 sde1
  0    0    1    1
  2    2    3    3
  4    4    5    5
  6    6    7    7
  8    8    9    9

Near layout with an odd number of disks and two replicas:

sda1 sdb1 sdc1 sde1 sdf1
  0    0    1    1    2
  2    3    3    4    4
  5    5    6    6    7
  7    8    8    9    9
  10   10   11   11   12

9.2.2.2 Far layout #

  

The far layout stripes data over the early part of all drives, then stripes a second copy of the data over the later part of all drives, making sure that all copies of a block are on different drives. The second set of values starts halfway through the component drives.

With a far layout, the read performance of the complex RAID 10 is similar to a RAID 0 over the full number of drives, but write performance is substantially slower than a RAID 0 because there is more seeking of the drive heads. It is best used for read-intensive operations such as for read-only file servers.

The speed of the RAID 10 for writing is similar to other mirrored RAID types, like RAID 1 and RAID 10 using near layout, as the elevator of the file system schedules the writes in a more optimal way than raw writing. Using RAID 10 in the far layout is well suited for mirrored writing applications.

Far layout with an even number of disks and two replicas:

sda1 sdb1 sdc1 sde1
  0    1    2    3
  4    5    6    7
  . . .
  3    0    1    2
  7    4    5    6

Far layout with an odd number of disks and two replicas:

sda1 sdb1 sdc1 sde1 sdf1
  0    1    2    3    4
  5    6    7    8    9
  . . .
  4    0    1    2    3
  9    5    6    7    8

9.2.2.3 Offset layout #

  

The offset layout duplicates stripes so that the multiple copies of a given chunk are laid out on consecutive drives and at consecutive offsets. Effectively, each stripe is duplicated and the copies are offset by one device. This should give similar read characteristics to a far layout if a suitably large chunk size is used, but without as much seeking for writes.

Offset layout with an even number of disks and two replicas:

sda1 sdb1 sdc1 sde1
  0    1    2    3
  3    0    1    2
  4    5    6    7
  7    4    5    6
  8    9   10   11
 11    8    9   10

Offset layout with an odd number of disks and two replicas:

sda1 sdb1 sdc1 sde1 sdf1
  0    1    2    3    4
  4    0    1    2    3
  5    6    7    8    9
  9    5    6    7    8
 10   11   12   13   14
 14   10   11   12   13

9.2.2.4 Specifying the number of replicas and the layout with YaST and mdadm #

  

The number of replicas and the layout is specified as Parity Algorithm in YaST or with the --layout parameter for mdadm. The following values are accepted:

nN

Specify n for near layout and replace N with the number of replicas. n2 is the default that is used when not configuring layout and the number of replicas.

fN

Specify f for far layout and replace N with the number of replicas.

oN

Specify o for offset layout and replace N with the number of replicas.

Note: Number of replicas

YaST automatically offers a selection of all possible values for the Parity Algorithm parameter.

Increasing the size of a software RAID involves the following tasks in the given order: increase the size of all partitions the RAID consists of, increase the size of the RAID itself and, finally, increase the size of the file system.

Warning: Potential data loss

If a RAID does not have disk fault tolerance, or it is simply not consistent, data loss results if you remove any of its partitions. Be very careful when removing partitions, and ensure that you have a backup of your data available.