安装和设置快速入门
SUSE Linux Enterprise High Availability Extension 15 SP4

# 安装和设置快速入门 #

出版日期：2025 年 12 月 11 日

本文档会指导您使用 crm 外壳提供的引导脚本完成最基本的双节点群集的设置。其中包括将虚拟 IP 地址配置为群集资源，以及在共享储存上使用 SBD 作为节点屏蔽机制。

作者: Tanja Roth、Thomas Schraitle
修订历史: 安装和设置快速入门

## 1 使用情形 #

通过本文档中介绍的过程可完成具有以下属性的双节点群组的基本设置：

- 两个节点：`alice`（IP：`192.168.1.1`）和 `bob`（IP：`192.168.1.2`），两者之间通过网络彼此相连。

- 一个浮动虚拟 IP 地址 (`192.168.2.1`)，客户端可通过该地址连接到服务，而无需考虑运行该服务的物理节点。

- 一台共享存储设备，用作 SBD 屏蔽机制。可避免节点分裂的情况。

- 当活动的主机发生故障（主动/被动设置）时，资源从一个节点故障转移至另一个节点。

使用引导脚本设置群集后，我们将使用图形 Hawk2 来监视群集。它是群集管理工具之一，随附于 SUSE® Linux Enterprise High Availability Extension 中。为了对资源故障转移是否有效进行最基本的测试，我们会将一个节点置于待机模式，并检查虚拟 IP 地址是否已迁移到另一个节点。

您可以使用该双节点群集进行测试，或以它为最小群集配置，稍后再行扩展。将该群集用于生产环境之前，请根据自己的要求进行修改。

## 2 系统要求 #

本章介绍第 1 节中所述案例的重要系统要求。要将群集调整为可用于生产环境，请参考Book “管理指南”, Chapter 2 “系统要求和建议”中的完整列表。

### 2.1 硬件要求 #

服务器

  两台安装有第 2.2 节 “软件需求”中指定的软件的服务器。

  服务器可以是裸机，也可以是虚拟机。两台服务器不需要使用相同的硬件（内存、磁盘空间等），但它们的体系结构必须相同。不支持跨平台群集。

通讯通道

  每个群集节点至少有两个 TCP/IP 通讯媒体。网络设备必须支持您要用于群集通讯的通讯方式：多路广播或单路广播。通讯媒体应支持 100 Mbit/s 或更高的数据传送速度。对于受支持的群集设置，要求有两个或更多冗余通讯路径。这可通过以下方式实现：

  - 网络设备绑定（首选）。

  - Corosync 中的另一个通讯通道。

节点屏蔽/STONITH

  为了避免发生“节点分裂”的情况，群集需要有节点屏蔽机制。在节点分裂的情况下，群集节点会由于硬件或软件故障或者网络连接断开而分成两个或更多互不相识的组。而屏蔽机制会隔离存在问题的节点（通常的做法是重设置该节点或关闭其电源）。这也称为 STONITH（“Shoot the other node in the head”，关闭其他节点）。节点屏蔽机制可以是物理设备（电源开关），也可以是 SBD（按磁盘 STONITH）等机制再结合检查包。使用 SBD 需要有共享存储。

### 2.2 软件需求 #

将加入群集的所有节点上都至少需安装以下模块和扩展：

- Basesystem Module 15 SP4

- Server Applications Module 15 SP4

- SUSE Linux Enterprise High Availability Extension 15 SP4

### 2.3 其他要求和建议 #

时间同步

  群集节点必须同步到群集外的 NTP 服务器。自 SUSE Linux Enterprise High Availability Extension 15 起，采用 chrony 作为 NTP 的默认实施。有关详细信息，请参见 《SUSE Linux Enterprise Server 15 SP4 管理指南》。

  如果节点未同步，群集可能无法正常运作。此外，日志文件和群集报告在不进行同步的情况下也很难进行分析。如果使用引导脚本，而 NTP 尚未配置，则系统会提出警告。

主机名和 IP 地址

  - 使用静态 IP 地址。

  - 在 `/etc/hosts` 文件中列出了所有群集节点，包括其完全限定的主机名和简短主机名。群集成员必须能够按名称找到彼此。如果名称不可用，则将无法进行群集内部通讯。

SSH

  所有群集节点都必须能通过 SSH 相互访问。`crm report`（用于查错）等工具和 Hawk2 的历史记录浏览器要求节点之间采用无口令 SSH 访问方式，否则它们只能从当前节点收集数据。

  如果使用引导脚本设置群集，系统会自动建立并复制 SSH 密钥。

## 3 引导脚本概述 #

以下命令可执行只需要极少时间和手动干预的引导脚本。

- 使用 `crm cluster init` 可定义群集通讯所需的基本参数。这将为您提供一个运行中的单节点群集。

- 使用 `crm cluster join` 可向群集添加更多节点。

- 使用 `crm cluster remove` 从群集中去除节点。

所有引导脚本都会记录到 `/var/log/crmsh/crmsh.log` 中。有关引导过程的任何细节都可以查看此文件。在引导过程中设置的任何选项都可稍后使用 YaST 群集模块进行修改。有关详细信息，请参见 Book “管理指南”, Chapter 4 “使用 YaST 群集模块”。

引导脚本 `crm cluster init` 会检查并配置以下组件：

NTP

  如果 NTP 未配置为在引导时启动，将显示一条消息。自 SUSE Linux Enterprise High Availability Extension 15 起，采用 chrony 作为 NTP 的默认实施。

SSH

  它会为群集节点之间的无口令登录建立 SSH 密钥。

Csync2

  它会配置 Csync2，让其在群集中的所有节点上复制配置文件。

Corosync

  它会配置群集通讯系统。

SBD/检查包

  它会检查是否存在检查包，并询问您是否要将 SBD 配置为节点屏蔽机制。

虚拟浮动 IP

  它会询问您是否要配置用于使用 Hawk2 进行群集管理的虚拟 IP 地址。

防火墙

  它会开启群集通讯所需的防火墙端口。

群集名称

  它会为群集定义名称，默认为 `hacluster`。群集名称是可选的，对 Geo 群集最有用。群集名称通常会反映位置，以便您更容易区分 Geo 群集内部的站点。

QDevice/QNetd

  本指南不介绍此设置。要使用 QNetd 服务器，可以按Book “管理指南”, Chapter 12 “QDevice 和 QNetd”中所述使用引导脚本进行设置。

## 4 安装 SUSE Linux Enterprise High Availability Extension #

使用 High Availability Extension 配置和管理群集的软件包内含在 `High Availability` 安装软件集（在命令行上以 `sles_ha` 指定）中。只有在将 SUSE Linux Enterprise High Availability Extension 安装为 SUSE® Linux Enterprise Server 的扩展后，此软件集才可用。

有关如何安装扩展的信息，请参见《SUSE Linux Enterprise Server 15 SP4 部署指南》。

过程 1︰ 安装 `High Availability` 软件集 #

如果尚未安装该软件集，请按如下所述操作：

1. 通过命令行使用 Zypper 来安装：

       root # zypper install -t pattern ha_sles

2. 在将包含在群集中的所有计算机上安装 High Availability 软件集。

   注意：在所有参与节点上安装软件包

   如果要自动安装 SUSE Linux Enterprise Server 15 SP4 和 SUSE Linux Enterprise High Availability Extension 15 SP4，请使用 AutoYaST 克隆现有节点。有关详细信息，请参见Book “管理指南”, Chapter 3 “安装 High Availability Extension”, Section 3.2 “使用 AutoYaST 进行批量安装和部署”。

3. 在 SUSE Customer Center 中注册计算机。有关详细信息，请参见 《SUSE Linux Enterprise Server 15 SP4 升级指南》。

## 5 使用 SBD 作为屏蔽机制 #

如果有 SAN（储存区域网络）等共享储存，可以使用它们来避免节点分裂的情况。要实现此目的，请配置 SBD 作为节点屏蔽机制。SBD 使用检查包支持和 `external/sbd` STONITH 资源代理。

### 5.1 SBD 的要求 #

在使用 `crm cluster init` 设置第一个节点的过程中，您可以决定是否使用 SBD。如果要使用，需要输入共享存储设备的路径。默认情况下，`crm cluster init` 将自动在设备上创建一个用于 SBD 的小分区。

要使用 SBD，必须符合以下要求：

- 群集中的所有节点上，共享存储设备的路径都必须永久且一致。使用稳定的设备名称，如 `/dev/disk/by-id/dm-uuid-part1-mpath-abcedf12345`。

- SBD 设备不得使用基于主机的 RAID、LVM2，也不能位于 DRBD* 实例上。

有关如何设置共享储存的细节，请参见 《SUSE Linux Enterprise Server 15 SP4 储存管理指南》。

### 5.2 对 SBD 启用 Softdog 检查包 #

在 SUSE Linux Enterprise Server 中，默认会启用内核中的检查包支持：本产品随附了多个内核模块，可提供特定于硬件的检查包驱动程序。High Availability Extension 使用 SBD 守护程序作为“供给”检查包的软件组件。

以下过程使用 `softdog` 检查包。

重要：Softdog 限制

Softdog 驱动程序假设至少有一个 CPU 仍然在运行。如果所有 CPU 均已阻塞，则 softdog 驱动程序中应该重引导系统的代码永远都不会执行。相反地，即使所有 CPU 均已阻塞，硬件检查包也仍然会继续工作。

强烈建议您在生产环境中使用群集之前，以最适合您硬件的相应硬件模块替换 `softdog` 模块。

不过，如果没有与您的硬件匹配的检查包，则可以将 `softdog` 用作内核检查包模块。

1. 如第 5.1 节 “SBD 的要求”中所述，建立永久性的共享存储。

2. 启用 softdog 检查包：

       root # echo softdog > /etc/modules-load.d/watchdog.conf
       root # systemctl restart systemd-modules-load

3. 测试 softdog 模块是否已正确装载：

       root # lsmod | grep dog
       softdog                16384  1

强烈建议您测试 SBD 屏蔽机制是否能正常阻止节点分裂情况。此类测试可以通过阻止 Corosync 群集通讯来完成。

## 6 设置第一个节点 #

使用 `crm cluster init` 脚本设置第一个节点。此操作所需的时间和手动干预都极少。

过程 2︰ 使用 `crm cluster init` 设置第一个节点 (`alice`) #

1. 以 `root` 身份登录到要用作群集节点的物理机或虚拟机。

2. 通过执行以下命令启动引导脚本：

       root # crm cluster init --name CLUSTERNAME

   使用一个有意义的名称（例如群集的地理位置 `amsterdam`）替换 CLUSTERNAME 占位符。若要在稍后创建 Geo 群集，这样做特别有用，因为它使站点的识别变得简单。

   如果您需要使用多路广播而不是单路广播（默认设置）来进行群集通讯，请使用选项 `--multicast`（或 `-U`）。

   该脚本会检查是否存在 NTP 配置和硬件检查包服务。此操作会生成用于 SSH 存取和 Csync2 同步的公共和私用 SSH 密钥，并启动相应的服务。

3. 配置群集通讯层 (Corosync)：

   1. 输入要绑定的网络地址。默认情况下，脚本将建议使用网络地址 `eth0`。也可以输入其他网络地址，例如地址 `bond0`。

   2. 接受建议的端口 (`5405`) 或输入其他端口。

4. 将 SBD 设置为节点屏蔽机制：

   1. 输入 `y` 确认您要使用 SBD。

   2. 输入要用于 SBD 的块设备分区的永久路径。请参见第 5 节 “使用 SBD 作为屏蔽机制”。该路径必须在群集中的所有节点中都一致。

5. 配置虚拟 IP 地址，以使用 Hawk2 进行群集管理。（稍后，我们将使用此虚拟 IP 资源来测试故障迁移是否成功）。

   1. 输入 `y` 确认您要配置虚拟 IP 地址。

   2. 为 Hawk2 的管理 IP 输入未使用的 IP 地址：`192.168.2.1`

      然后，您便可以连接到该虚拟 IP 地址，而无需使用 Hawk2 登录单个群集节点。

最后，该脚本将启动 Pacemaker 服务以使群集联机，并启用 Hawk2。要用于 Hawk2 的 URL 将显示在屏幕上。

现在必须有一个运行中的单节点群集。要查看其状态，请执行以下操作：

过程 3︰ 登录 Hawk2 Web 界面 #

1. 在任何计算机上，启动 Web 浏览器并确保 JavaScript 和 cookie 已启用。

2. 输入运行 Hawk Web 服务的任何群集节点的 IP 地址或主机名作为 URL。或者，输入在过程 2 “使用 `crm cluster init` 设置第一个节点 (`alice`)”的步骤 5中配置的虚拟 IP 地址的地址：

       https://HAWKSERVER:7630/

   注意：证书警告

   当您首次尝试访问 URL 时如果显示证书警告，则表示使用了自我签名证书。默认情况下，自我签名证书不被视为可信证书。

   请向您的群集操作员询问证书细节，以校验该证书。

   要继续，可在浏览器中添加例外，以绕过警告。

3. 在 Hawk2 登录屏幕上，输入引导过程中建立的使用者的用户名和口令（用户 `hacluster`，口令 `linux`）。

   重要：安全密码

   尽快用安全密码替换默认密码：

       root # passwd hacluster

4. 单击登录。登录之后，Hawk2 Web 界面默认会显示“状态”屏幕，其中包含当前群集状态一览：

   图 1︰ Hawk2 中单节点群集的状态 #

## 7 添加第二个节点 #

如果您设置了单节点群集并且该群集正在运行，则可按过程 4 中所述使用 `crm cluster join` 引导脚本添加第二个群集节点。该脚本只需访问一个现有群集节点即可在当前计算机上自动完成基本设置。有关细节，请参见 `crm cluster join` 手册页。

引导脚本负责更改特定于双节点群集的配置，例如 SBD、Corosync。

过程 4︰ 使用 `crm cluster join` 添加第二个节点 (`bob`) #

1. 以 `root` 用户身份登录到将要加入群集的物理机或虚拟机。

2. 通过执行以下命令启动引导脚本：

       root # crm cluster join

   如果 NTP 未配置为在引导时启动，将显示一条消息。该脚本还会检查硬件检查包设备（在要配置 SBD 时，此类设备非常重要）。如果不存在此类设备，将会向您发出警告。

3. 如果仍要继续，系统将提示您输入现有节点的 IP 地址。输入第一个节点 (`192.168.1.1` `alice`) 的 IP 地址。

4. 如果尚未配置两台计算机之间的无密码 SSH 访问，系统将提示您输入现有节点的 `root` 密码。

   登录到指定节点后，脚本将会复制 Corosync 配置，配置 SSH、Csync2，并使当前计算机作为新群集节点联机。除此之外，还将启动 Hawk2 所需的服务。

在 Hawk2 中检查群集状态。在 状态 › 节点 下，应该会看到状态为绿色的两个节点（请参见图 2 “双节点群集的状态”）。

图 2︰ 双节点群集的状态 #

## 8 测试群集 #

第 8.1 节 “测试资源故障转移”是一项检查当群集中当前运行资源的节点被设置为`待机`时，群集是否将虚拟 IP 地址移动到另一个节点的简单测试。

不过，实际的测试涉及特定的使用案例和情形，包括测试用于避免出现节点分裂情况的屏蔽机制。如果尚未正确设置屏蔽机制，群集将无法正常运作。

将该群集用于生产环境之前，请根据自己的用例或者使用 `ha-cluster-preflight-check` 脚本对群集进行全面测试。

### 8.1 测试资源故障转移 #

请使用以下快速测试过程检查资源故障转移：

过程 5︰ 测试资源故障转移 #

1. 打开终端并 ping `192.168.2.1`（您的虚拟 IP 地址）：

       root # ping 192.168.2.1

2. 如过程 3 “登录 Hawk2 Web 界面”中所述，登录到您的群集。

3. 在 Hawk2 状态 › 资源 中，检查运行该虚拟 IP 地址（资源 `admin_addr`）的节点。我们假设资源运行于 `alice`。

4. 将 `alice` 置于待机模式（请参见图 3 “处于待机模式的节点 `alice`”）。

   图 3︰ 处于待机模式的节点 `alice` #

5. 单击 状态 › 资源 。资源 `admin_addr` 已迁移到 `bob`。

在迁移期间，应该会看到针对虚拟 IP 地址的不间断的 ping 流。这表示该群集设置和浮动 IP 运作正常。使用 `ping` 取消 Ctrl–C 命令。

### 8.2 使用 ha-cluster-preflight-check 命令进行测试 #

命令 `ha-cluster-preflight-check` 会对群集运行标准化测试。它会触发群集故障，并校验配置以找出问题。在生产环境中使用群集之前，建议先使用此命令来确保一切符合预期。

该命令支持以下检查：

- 环境检查 `-e`/`--env-check`： 此项测试将会检查：

  - 主机名是否可解析？

  - 是否已启用并启动时间服务？

  - 是否为当前节点配置了检查包？

  - 是否已启用 `firewalld` 服务，并打开了群集相关的端口？

- 群集状态检查 `-c`/`--cluster-check`： 检查群集的不同状态和服务。此项测试将会检查：

  - 群集服务 (Pacemaker/Corosync) 是否已启用并正在运行？

  - 是否已启用 STONITH？它还会检查是否已配置并启动 STONITH 相关的资源。如果您配置了 SBD，SBD 服务是否已启动？

  - 群集是否具有仲裁？显示当前 DC 节点，以及处于联机、脱机和未清理状态的节点。

  - 是否存在已启动、已停止或有故障的资源？

- 节点分裂检查 `--split-brain-iptables`： 通过阻止 Corosync 端口来模拟节点分裂场景。检查是否可按预期屏蔽一个节点。

- 终止 SBD、Corosync 和 Pacemaker 的守护程序 `-kill-sbd`/`-kill-corosync`/`-kill-pacemakerd`： 运行此类测试后，可以在 `/var/lib/ha-cluster-preflight-check` 中找到报告。该报告包括测试案例说明、操作日志，以及对可能结果的说明。

- 屏蔽节点检查 `--fence-node`：屏蔽从命令行传递的特定节点。

例如，要测试环境，请运行：

    root # crm_mon -1
    Stack: corosync
    Current DC: alice (version ...) - partition with quorum
    Last updated: Fri Mar 03 14:40:21 2020
    Last change: Fri Mar 03 14:35:07 2020 by root via cibadmin on alice

    2 nodes configured
    1 resource configured

    Online: [ alice bob ]
    Active resources:

     stonith-sbd    (stonith:external/sbd): Started alice

    root # ha-cluster-preflight-check -e
    [2020/03/20 14:40:45]INFO: Checking hostname resolvable [Pass]
    [2020/03/20 14:40:45]INFO: Checking time service [Fail]
     INFO: chronyd.service is available
     WARNING: chronyd.service is disabled
     WARNING: chronyd.service is not active
    [2020/03/20 14:40:45]INFO: Checking watchdog [Pass]
    [2020/03/20 14:40:45]INFO: Checking firewall [Fail]
     INFO: firewalld.service is available
     WARNING: firewalld.service is not active

可以在 `/var/log/ha-cluster-preflight-check.log` 中检查结果。

## 9 更多信息 #

https://documentation.suse.com/sle-ha/ 上提供了更多有关此产品的文档。有关其他配置和管理任务，请参见详尽的 《管理指南》。

## 10 法律声明 #

版权所有 © 2006– 2025 SUSE LLC 和贡献者。保留所有权利。

根据 GNU 自由文档许可 (GNU Free Documentation License) 版本 1.2 或（根据您的选择）版本 1.3 中的条款，在此授予您复制、分发和/或修改本文档的权限；本版权声明和许可附带不可变部分。许可版本 1.2 的副本包含在题为“GNU Free Documentation License”的部分。

有关 SUSE 商标，请参见 http://www.suse.com/company/legal/。所有其他第三方商标分别为相应所有者的财产。商标符号（®、™ 等）代表 SUSE 及其关联公司的商标。星号 (*) 代表第三方商标。

本指南力求涵盖所有细节，但这不能确保本指南准确无误。SUSE LLC 及其关联公司、作者和译者对于可能出现的错误或由此造成的后果皆不承担责任。

# A GNU licenses #

This appendix contains the GNU Free Documentation License version 1.2.

## GNU Free Documentation License #

Copyright (C) 2000, 2001, 2002 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

##### 0. PREAMBLE #

The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or non-commercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

##### 1. APPLICABILITY AND DEFINITIONS #

This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

##### 2. VERBATIM COPYING #

You may copy and distribute the Document in any medium, either commercially or non-commercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

##### 3. COPYING IN QUANTITY #

If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

##### 4. MODIFICATIONS #

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.

3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

4. Preserve all the copyright notices of the Document.

5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

8. Include an unaltered copy of this License.

9. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

11. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

13. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version.

14. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section.

15. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

##### 5. COMBINING DOCUMENTS #

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements".

##### 6. COLLECTIONS OF DOCUMENTS #

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

##### 7. AGGREGATION WITH INDEPENDENT WORKS #

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

##### 8. TRANSLATION #

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.

##### 9. TERMINATION #

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

##### 10. FUTURE REVISIONS OF THIS LICENSE #

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See https://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

##### ADDENDUM: How to use this License for your documents #

    Copyright (c) YEAR YOUR NAME.
    Permission is granted to copy, distribute and/or modify this document
    under the terms of the GNU Free Documentation License, Version 1.2
    or any later version published by the Free Software Foundation;
    with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
    A copy of the license is included in the section entitled “GNU
    Free Documentation License”.

If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with...Texts.” line with this:

    with the Invariant Sections being LIST THEIR TITLES, with the
    Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.

If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.