Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / SUSE Linux Enterprise Server Documentation / SMT Guide / Mirroring Repositories on the SMT Server
Applies to SUSE Linux Enterprise Server 12 SP5

4 Mirroring Repositories on the SMT Server

You can mirror the installation and update repositories on the SMT server. This way, you do not need to download updates on each machine, which saves time and bandwidth.

Important
Important: SUSE Linux Enterprise Server 9 Repositories

As SUSE Linux Enterprise Server 9 is no longer supported, SMT does not mirror SUSE Linux Enterprise Server 9 repositories.

4.1 Mirroring Credentials

Before you create a local mirror of the repositories, you need appropriate organization credentials. You can obtain the credentials from SUSE Customer Center.

To get the credentials from SUSE Customer Center, follow these steps:

  1. Visit SUSE Customer Center at http://scc.suse.com and log in.

  2. If you are a member of multiple organizations, select the organization you want to work with from the drop-down box in the top-right corner.

  3. Click Proxies in the top menu.

  4. Switch to the Mirroring credentials section.

  5. To see the password, click the Eye iconicon.

The obtained credentials should be set with the YaST SMT Server Configuration module or added directly to the /etc/smt.conf file. For more information about the /etc/smt.conf file, see Section 8.2.1, “/etc/smt.conf”.

Tip
Tip: Merging Multiple Organization Site Credentials

SMT can only work with one mirror credential at a time. Multiple credentials are not supported. When a customer creates a new company, this generates a new mirror credential. This is not always convenient, as some products are available via the first set and other products via the second set. To request a merge of credentials, contact your account manager.

4.2 Managing Software Repositories with SMT Command Line Tools

This section describes tools and procedures for viewing information about software repositories available through SMT, configuring these repositories, and setting up custom repositories on the command line. For details on the YaST SMT Server Management module, see Chapter 5, Managing Repositories with YaST SMT Server Management.

4.2.1 Updating the Local SMT Database

The local SMT database needs to be updated periodically with the information downloaded from SUSE Customer Center. These periodic updates can be configured with the SMT Management module, as described in Section 3.5, “Setting the SMT Job Schedule with YaST”.

To update the SMT database manually, use the smt-sync command. For more information about the smt-sync command, see Section 8.1.2.7, “smt-sync”.

4.2.2 Enabled Repositories and Repositories That Can Be Mirrored

The database installed with SMT contains information about all software repositories available on SUSE Customer Center. However, the used mirror credentials determine which repositories can really be mirrored. For more information about getting and setting organization credentials, see Section 4.1, “Mirroring Credentials”.

Repositories that can be mirrored have the MIRRORABLE flag set in the repositories table in the SMT database. That a repository can be mirrored does not mean that it needs to be mirrored. Only repositories with the DOMIRROR flag set in the SMT database will be mirrored. For more information about configuring which repositories should be mirrored, see Section 4.2.4, “Selecting Repositories to Be Mirrored”.

4.2.3 Getting Information about Repositories

Use the smt-repos command to list available software repositories and additional information. Using this command without any options lists all available repositories, including repositories that cannot be mirrored. In the first column, the enabled repositories (repositories set to be mirrored) are marked with Yes. Disabled repositories are marked with No. The other columns show ID, type, name, target, and description of the listed repositories. The last columns show whether the repository can be mirrored and whether staging is enabled.

Use the --verbose option, to get additional information about the URL of the repository and the path it will be mirrored to.

The repository listing can be limited to the repositories that can be mirrored or to the repositories that are enabled. To list the repositories that can be mirrored, use the -m or --only-mirrorable option: smt-repos -m.

To list only enabled repositories, use the -o or --only-enabled option: smt-repos -o (see Example 4.1, “Listing All Enabled Repositories”).

Example 4.1: Listing All Enabled Repositories
tux:~ # smt-repos -o
.---------------------------------------------------------------------------------------------------------------------.
| Mirr| ID | Type | Name                    | Target        | Description                             | Can be M| Stag|
+-----+----+------+-------------------------+---------------+-----------------------------------------+---------+-----+
| Yes |  1 | zypp | ATI-Driver-SLE11-SP2    | --            | ATI-Driver-SLE11-SP2                    | Yes     | Yes |
| Yes |  2 | zypp | nVidia-Driver-SLE11-SP2 | --            | nVidia-Driver-SLE11-SP2                 | Yes     | No  |
| Yes |  3 | nu   | SLED11-SP2-Updates      | sle-11-x86_64 | SLED11-SP2-Updates for sle-11-x86_64    | Yes     | No  |
| Yes |  4 | nu   | SLES11-SP1-Updates      | sle-11-x86_64 | SLES11-SP1-Updates for sle-11-x86_64    | Yes     | Yes |
| Yes |  5 | nu   | SLES11-SP2-Core         | sle-11-x86_64 | SLES11-SP2-Core for sle-11-x86_64       | Yes     | No  |
| Yes |  6 | nu   | SLES11-SP2-Updates      | sle-11-i586   | SLES11-SP2-Updates for sle-11-i586      | Yes     | No  |
| Yes |  7 | nu   | WebYaST-Testing-Updates | sle-11-i586   | WebYaST-Testing-Updates for sle-11-i586 | Yes     | No  |
'-----+----+------+-------------------------+---------------+-----------------------------------------+---------+-----'

You can also list only repositories with a specific name or show information about a repository with a specific name and target. To list repositories with a particular name, use the smt-repos REPOSITORY_NAME command. To show information about a repository with a specific name and target, use the smt-repos repository_name TARGET command.

To get a list of installation repositories from remote, see Section 9.7, “Listing Accessible Repositories”.

4.2.4 Selecting Repositories to Be Mirrored

Only enabled repositories can be mirrored. In the database, the enabled repositories have the DOMIRROR flag set. Repositories can be enabled or disabled using the smt-repos command.

To enable one or more repositories, follow these steps:

  1. To enable all repositories that can be mirrored or to choose one repository from the list of all repositories, run the smt-repos -e command.

    You can limit the list of repositories by using the relevant options. To limit the list to the repositories that can be mirrored, use the -m option: smt-repos -m -e. To limit the list to the repositories with a specific name, use the smt-repos -e REPOSITORY_NAME command. To list a repository with a specific name and target, use the smt-repos -e REPOSITORY_NAME TARGET command.

    To enable all repositories belonging to a specific product, use the --enable-by-prod or -p option, followed by the name of the product and optionally the version, architecture, and release:

    smt-repos -p product[,version[,architecture[,release]]]

    For example, to enable all repositories belonging to SUSE Linux Enterprise Server 10 SP3 for PowerPC architecture, use the smt-repos -p SUSE-Linux-Enterprise-Server-SP3,10,ppc command. The list of known products can be obtained with the smt-list-products command.

    Tip
    Tip: Installer Self-Update Repository

    SMT supports mirroring the installer self-update repository (find more information in Section 6.4.1, “Self-Update Process”). If you need to provide the self-update repository, identify and enable it, for example:

    $ smt-repos -m | grep Installer
    $ smt-repos -e SLES12-SP2-Installer-Updates sle-12-x86_64
  2. If more than one repository is listed, choose the one you want to enable: specify its ID listed in the repository table and press Enter. If you want to enable all the listed repositories, use a and press Enter.

To disable one or more repositories, follow these steps:

  1. To disable all enabled repositories or just choose one repository from the list of all repositories, run the smt-repos -d command.

    To choose the repository to be disabled from a shorter list, or to disable all repositories from a limited group, use any of the available options to limit the list of repositories. To limit the list to the enabled repositories, use the -o option: smt-repos -o -d. To limit the list to repositories with a particular name, use the smt-repos -d REPOSITORY_NAME command. To show a repository with a specific name and target, use the smt-repos -d REPOSITORY_NAME TARGET command.

  2. If more than one repository is listed, choose which one you want to disable: specify its ID listed in the repository table and press Enter. If you want to disable all the listed repositories, use a and press Enter.

4.2.5 Deleting Mirrored Repositories

You can delete mirrored repositories that are no longer used. If you delete a repository, it will be physically removed from the SMT storage area.

Use the smt-repos --delete command to delete a repository with a specific name. To delete the repository in a namespace, specify the --namespace DIRNAME option.

The --delete option lists all repositories. You can delete the specified repositories by entering the ID number or the name and target. To delete all repositories, enter a.

Note
Note: Detecting Repository IDs

Every repository has an SHA-1 hash that you can use as an ID. You can get the repository's hash by calling smt-repos -v.

4.2.6 Mirroring Custom Repositories

SMT also makes it possible to mirror repositories that are not available at the SUSE Customer Center. These repositories are called custom repositories, and they can be mirrored using the smt-setup-custom-repos command. It is also possible to delete custom repositories.

When adding a new custom repository, the smt-setup-custom-repos command inserts a new record in the database and sets the mirror flag to true. You can disable mirroring later, if necessary.

To set up a custom repository to be available through SMT, follow these steps:

  1. If you do not know the ID of the product the new repositories should belong to, use smt-list-products to get the ID. For the description of the smt-list-products, see Section 8.1.2.4, “smt-list-products”.

  2. Run

    smt-setup-custom-repos --productid PRODUCT_ID \
    --name REPOSITORY_NAME --exturl REPOSITORY_URL

    PRODUCT_ID is the ID of the product the repository belongs to, REPOSITORY_NAME is the name of the repository, and REPOSITORY_URL is the URL of the repository. If the added repository needs to be available for more than one product, specify the IDs of all products that should use the added repository.

    For example, the following command sets My repository available at http://example.com/My_repository to the products with the IDs 423, 424, and 425:

    smt-setup-custom-repos --productid 423 --productid 424 \
    --productid 425 --name 'My_repository' \
    --exturl 'http://example.com/My_repository'
Note
Note: Mirroring Unsigned Repositories

By default, SUSE Linux Enterprise 10 does not allow the use of unsigned repositories. So if you want to mirror unsigned repositories and use them on client machines, be aware that the package installation tool—YaST or zypper—will ask you whether to use repositories that are not signed.

To remove an existing custom repository from the SMT database, use smt-setup-custom-repos --delete ID, where ID is the ID of the repository to be removed.

4.3 The Structure of /srv/www/htdocs for SLE 11

The path to the directory containing the mirror is set by the MirrorTo option in the /etc/smt.conf configuration file. For more information about /etc/smt.conf, see Section 8.2.1, “/etc/smt.conf”. If the MirrorTo option is not set to the Apache htdocs directory /srv/www/htdocs/, the following links need to be created. If the directories already exist, they need to be removed prior to creating the link (the data in these directories will be lost). In the following examples, MIRRORTO needs to be replaced by the path the option MirrorTo is set to.

  • /srv/www/htdocs/repo/$RCE must point to MIRRORTO/repo/$RCE/

  • /srv/www/htdocs/repo/RPMMD must point to MIRRORTO/repo/RPMMD/

  • /srv/www/htdocs/repo/testing must point to MIRRORTO/repo/testing/

  • /srv/www/htdocs/repo/full must point to MIRRORTO/repo/full/

The directory specified using the MirrorTo option and the subdirectories listed above must exist. Files, directories, and links in /MIRRORTO must belong to the smt user and the www group.

Here is an example where the MirrorTo is set to /mirror/data:

l /srv/www/htdocs/repo/
total 16
lrwxrwxrwx 1 smt  www    22 Feb  9 14:23 $RCE -> /mirror/data/repo/$RCE/
drwxr-xr-x 4 smt  www  4096 Feb  9 14:23 ./
drwxr-xr-x 4 root root 4096 Feb  8 15:44 ../
lrwxrwxrwx 1 smt  www    23 Feb  9 14:23 RPMMD -> /mirror/data/repo/RPMMD/
lrwxrwxrwx 1 smt  www    22 Feb  9 14:23 full -> /mirror/data/repo/full/
drwxr-xr-x 2 smt  www  4096 Feb  8 11:12 keys/
lrwxrwxrwx 1 smt  www    25 Feb  9 14:23 testing -> /mirror/data/repo/testing/
drwxr-xr-x 2 smt  www  4096 Feb  8 14:14 tools/

The links can be created using the ln -s commands. For example:

cd /srv/www/htdocs/repo
for LINK in \$RCE RPMMD full testing; do
 ln -s /mirror/data/repo/${LINK}/ && chown -h smt.www ${LINK}
done
Important
Important: The /srv/www/htdocs/repo Directory

The /srv/www/htdocs/repo directory must not be a symbolic link.

Important
Important: Apache and Symbolic Links

By default Apache on SUSE Linux Enterprise Server is configured to not follow symbolic links. To enable symbolic links for /srv/www/htdocs/repo/ add the following snippet to /etc/apache2/default-server.conf (or the respective virtual host configuration in case you are running SMT on a virtual host):

<Directory "/srv/www/htdocs/repo">
 Options FollowSymLinks
</Directory>

After having made the change, test the syntax and reload the Apache configuration files to activate the change:

rcapache2 configtest && rcapache2 reload

4.4 The Structure of /srv/www/htdocs for SLE 12

The repository structure in the /srv/www/htdocs directory matches the structure specified in SUSE Customer Center. There are the following directories in the structure (selected examples, similar for other products and architectures):

repo/SUSE/Products/SLE-SDK/12/x86_64/product/

Contains the -POOL repository of SDK (the GA version of all packages).

repo/SUSE/Products/SLE-SDK/12/x86_64/product.license/

Contains EULA associated with the product.

repo/SUSE/Updates/SLE-SDK/12/x86_64/update/
repo/SUSE/Updates/SLE-SDK/12/s390x/update/
repo/SUSE/Updates/SLE-SERVER/12/x86_64/update/

Contain update repositories for respective products.

repo/full/SUSE/Updates/SLE-SERVER/12/x86_64/update/
repo/testing/SUSE/Updates/SLE-SERVER/12/x86_64/update/

Contain repositories created for staging of respective repositories.

4.5 Using the Test Environment

You can mirror repositories to a test environment instead of the production environment. The test environment can be used with a limited number of client machines before the tested repositories are moved to the production environment. The test environment can be run on the main SMT server.

The testing environment uses the same structure as the production environment, but it is located in the /srv/www/htdocs/repo/testing/ subdirectory.

To mirror a repository to the testing environment, you can use the Staging tab in the YaST SMT Management module, or the command smt-staging.

To register a client in the testing environment, follow these steps:

  1. De-register the client from the SMT server by running SUSEConnect --de-register on the client host.

  2. Modify /etc/SUSEConnect on the client machine as follows:

    namespace: testing
  3. Re-register the client host against SMT in order for the new namespace setting to take effect. See general information about registering SMT clients in Chapter 9, Configuring Clients to Use SMT.

To move the testing environment to the production environment, manually copy or move it using the cp -a or mv command.

You can enable staging for a repository in the Repositories tab of the SMT Management module or with the smt-repos command. The mirroring happens automatically to repo/full/.

If you have an SLE11-based update repository with patches, SMT tools can be used to manage them. Using these tools, you can select patches, create a snapshot and copy it into repo/testing/. After tests are finished, you can copy the contents of repo/testing into the production area /repo.

SLE10-based update repositories are not supported by SMT tools. Not all of these repositories support selective staging. In this case, you must mirror the complete package.

Recommended workflow:

customer center => repo/full => repo/testing, => repo/production

4.6 Testing and Filtering Update Repositories with Staging

You can test repositories on any clients using the smt-staging command before moving them to the production environment. You can select new update repositories to be installed on clients.

You can either use the smt-staging command or the YaST SMT Management module for staging. For more details, see Section 5.3, “Staging Repositories”.

SMT Staging Schema
Figure 4.1: SMT Staging Schema

Repositories with staging enabled are mirrored to the /MIRRORTO/repo/full subdirectory. This subdirectory is usually not used by your clients. Incoming new updates are not automatically visible to the clients before you get a chance to test them. Later, you can generate a testing environment of this repository, which goes to the /MIRRORTO/repo directory.

If you have an SLE 11-based update repository with patches, you can use SMT tools to manage them. Using these tools, you can select patches, create a snapshot and put it into repo/testing/. After tests are finished, you can put the content of repo/testing into the /repo production area called the default staging group. You can create additional staging groups as needed using the smt-staging creategroup command.

Note
Note: SLE 10-based Update Repositories

SLE 10-based update repositories are not supported by SMT tools. Not all of these repositories support selective staging. In this case, you need to mirror the complete package.

Enabling Staging

To enable or disable staging, use the smt-repos command with the --enable-staging or -s options:

smt-repos --enable-staging

You can enable the required repositories by entering the ID number or by entering the name and target. If you want to enable all repositories, enter a.

Generating Testing and Production Snapshots

To create the testing repository in the default staging group, run the following command:

smt-staging createrepo REPOSITORY_ID -–testing

You can then test the installation and functionality of the patches in testing clients. If testing was successful, create the production repository:

smt-staging createrepo REPOSITORY_ID --production

To create testing and production repositories in a named staging group, create the group and the repositories in this group:

smt-staging creategroup GROUPNAME TESTINGDIR PRODUCTIONDIR
smt-staging createrepo --group GROUPNAME REPOSITORY_ID -–testing
SMT-STAGING createrepo --group GROUPNAME REPOSITORY_ID -–production

This can be useful when you want to combine SLES11-SP1-Updates and SLES11-SP2-Updates of the sle-11-x86_64 architecture into one repository of a group:

smt-staging creategroup SLES11SP1-SP2-Up test-sp1-sp2 prod-sp1-sp2
smt-staging createrepo --group SLES11SP1-SP2-Up \
  SLES11-SP1-Updates sle-11-x86_64 --testing
smt-staging createrepo --group SLES11SP1-SP2-Up \
  SLES11-SP2-Updates sle-11-x86_64 --testing
smt-staging createrepo --group SLES11SP1-SP2-Up \
  SLES11-SP1-Updates sle-11-x86_64 --production
smt-staging createrepo --group SLES11SP1-SP2-Up \
  SLES11-SP2-Updates sle-11-x86_64 --production

Group names can contain the following characters: -_, a-z A-Z, and 0-9.

Filtering Patches

You can allow or forbid all or selected patches using the allow or forbid commands:

smt-staging forbid --patch ID
smt-staging forbid --category CATEGORYNAME
Signing Changed Repositories

Filtering one or more patches from a repository invalidates the original signature, and the repository needs to be signed again. The smt-staging createrepo command does that automatically, provided you configure the SMT server.

To enable signing of changed metadata, the admin needs to generate a new signing key. This can be done with GPG like this:

mkdir DIR
gpg --gen-key --homedir DIR
sudo mv DIR /var/lib/smt/.gnupg
sudo chown smt:users -R /var/lib/smt/.gnupg
sudo chmod go-rwx -R /var/lib/smt/.gnupg

The ID of the newly generated key can be obtained using the gpg --gen-key command. The ID must be added to the signingKeyID option in the /etc/smt.conf file.

At this point, the clients are not aware of the new key. Import the new key to clients during their registration as follows:

sudo -u smt gpg --homedir /var/lib/smt/.gnupg \
  --export -a SIGNING_KEYID \
  > /MIRRORTO/repo/keys/smt-signing-key.key

In this example, MIRRORTO is the base directory where repositories will be mirrored. After that, clients can import this key during the registration process.

Registering Clients in the Testing Environment

To register a client in the testing environment, follow these steps:

  1. De-register the client from the SMT server by running SUSEConnect --de-register on the client host.

  2. Modify /etc/SUSEConnect on the client machine as follows:

    namespace: testing
  3. Re-register the client host against SMT in order for the new namespace setting to take effect. See general information about registering SMT clients in Chapter 9, Configuring Clients to Use SMT.

4.7 Repository Preloading

Deploying multiple SMT servers can take time if each new SMT server must mirror the same repositories.

To save time when deploying new SMT servers, the repositories can be preloaded from another server or disk instead. To do this, follow these steps:

  1. Enable the repositories to be mirrored with the SMT, for example:

    smt-repos -e SLES12-Updates sle-12-x86_64
  2. Perform a dry run of smt-mirror to create the required repository directories:

    smt-mirror -d --dryrun -L /var/log/smt/smt-mirror.log

    The following directories are created based on the repository above and the default MirrorTo:

    /srv/www/htdocs/repo/repoindex.xml
    /srv/www/htdocs/repo/$RCE/SLES12-Updates/sle-12-x86_64/*
  3. Then copy the repositories from another SMT server, for example:

    rsync -av 'smt12:/srv/www/htdocs/repo/\$RCE/SLES12-Updates/sle-12-x86_64/' \
     '/srv/www/htdocs/repo/$RCE/SLES12-Updates/sle-12-x86_64/'
  4. To get the repository data fixed, run the following command:

    smt-mirror -d -L /var/log/smt/smt-mirror.log
Important
Important: Possible Error Messages

Errors, such as repomd.xml is the same, but repo is not valid. Start mirroring., are considered normal. They occur because the metadata of the preloaded repositories in the server's database remains incorrect until the initial mirror of the repositories has completed.