19 Precision Time Protocol #
For network environments, it is vital to keep the computer and other devices' clocks synchronized and accurate. There are several solutions to achieve this, for example the widely used Network Time Protocol (NTP) described in Chapter 26, Time Synchronization with NTP.
The Precision Time Protocol (PTP) is a protocol capable of sub-microsecond accuracy, which is better than what NTP achieves. PTP support is divided between the kernel and user space. The kernel in SUSE Linux Enterprise Server includes support for PTP clocks, which are provided by network drivers.
19.1 Introduction to PTP #
The clocks managed by PTP follow a master-slave hierarchy. The slaves are synchronized to their masters. The hierarchy is updated by the best master clock (BMC) algorithm, which runs on every clock. The clock with only one port can be either master or slave. Such a clock is called an ordinary clock (OC). A clock with multiple ports can be master on one port and slave on another. Such a clock is called a boundary clock (BC). The top-level master is called the grandmaster clock. The grandmaster clock can be synchronized with a Global Positioning System (GPS). This way disparate networks can be synchronized with a high degree of accuracy.
The hardware support is the main advantage of PTP. It is supported by various network switches and network interface controllers (NIC). While it is possible to use non-PTP enabled hardware within the network, having network components between all PTP clocks PTP hardware enabled achieves the best possible accuracy.
19.1.1 PTP Linux Implementation #
On SUSE Linux Enterprise Server, the implementation of PTP is provided by the
linuxptp package. Install it with
zypper install linuxptp. It includes the
ptp4l and phc2sys programs for
clock synchronization. ptp4l implements the PTP
boundary clock and ordinary clock. When hardware time stamping is
enabled, ptp4l synchronizes the PTP hardware clock to
the master clock. With software time stamping, it synchronizes the
system clock to the master clock. phc2sys is needed
only with hardware time stamping to synchronize the system clock to the
PTP hardware clock on the network interface card (NIC).
19.2 Using PTP #
19.2.1 Network Driver and Hardware Support #
PTP requires that the used kernel network driver supports either
software or hardware time stamping. Moreover, the NIC must support time
stamping in the physical hardware. You can verify the driver and NIC
time stamping capabilities with ethtool:
ethtool -T eth0
Time stamping parameters for eth0:
Capabilities:
hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 0
Hardware Transmit Timestamp Modes:
off (HWTSTAMP_TX_OFF)
on (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
none (HWTSTAMP_FILTER_NONE)
all (HWTSTAMP_FILTER_ALL)Software time stamping requires the following parameters:
SOF_TIMESTAMPING_SOFTWARE SOF_TIMESTAMPING_TX_SOFTWARE SOF_TIMESTAMPING_RX_SOFTWARE
Hardware time stamping requires the following parameters:
SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_TX_HARDWARE SOF_TIMESTAMPING_RX_HARDWARE
19.2.2 Using ptp4l #
ptp4l uses hardware time stamping by default. As
root, you need to specify the network interface capable of
hardware time stamping with the -i option. The
-m tells ptp4l to print its output
to the standard output instead of the system's logging facility:
ptp4l -m -i eth0 selected eth0 as PTP clock port 1: INITIALIZING to LISTENING on INITIALIZE port 0: INITIALIZING to LISTENING on INITIALIZE port 1: new foreign master 00a152.fffe.0b334d-1 selected best master clock 00a152.fffe.0b334d port 1: LISTENING to UNCALIBRATED on RS_SLAVE master offset -25937 s0 freq +0 path delay 12340 master offset -27887 s0 freq +0 path delay 14232 master offset -38802 s0 freq +0 path delay 13847 master offset -36205 s1 freq +0 path delay 10623 master offset -6975 s2 freq -30575 path delay 10286 port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED master offset -4284 s2 freq -30135 path delay 9892
The master offset value represents the measured
offset from the master (in nanoseconds).
The s0, s1, s2
indicators show the different states of the clock servo:
s0 is unlocked,
s1 is clock step, and s2 is
locked. If the servo is in the locked state (s2), the
clock will not be stepped (only slowly adjusted) if the
pi_offset_const option is set to a negative value in
the configuration file (see man 8 ptp4l for more
information).
The freq value represents the frequency
adjustment of the clock (in parts per billion, ppb).
The path delay value represents the estimated delay of the
synchronization messages sent from the master (in nanoseconds).
Port 0 is a Unix domain socket used for local PTP management.
Port 1 is the eth0 interface.
INITIALIZING, LISTENING,
UNCALIBRATED and SLAVE are examples of
port states which change on INITIALIZE,
RS_SLAVE, and MASTER_CLOCK_SELECTED
events. When the port state changes from UNCALIBRATED
to SLAVE, the computer has successfully synchronized with
a PTP master clock.
You can enable software time stamping with the -S
option.
ptp4l -m -S -i eth3
You can also run ptp4l as a service:
systemctl start ptp4l
In this case, ptp4l reads its options from the
/etc/sysconfig/ptp4l file. By default, this file
tells ptp4l to read the configuration options from
/etc/ptp4l.conf. For more information on
ptp4l options and the configuration file settings,
see man 8 ptp4l.
To enable the ptp4l service permanently, run the
following:
systemctl enable ptp4l
To disable it, run
systemctl disable ptp4l
19.2.3 ptp4l Configuration File #
ptp4l can read its configuration from an optional
configuration file. As no configuration file is used by default, you
need to specify it with -f.
ptp4l -f /etc/ptp4l.conf
The configuration file is divided into sections. The global section
(indicated as [global]) sets the program options,
clock options and default port options. Other sections are port
specific, and they override the default port options. The name of the
section is the name of the configured port—for example,
[eth0]. An empty port section can be used to replace
the command line option.
[global] verbose 1 time_stamping software [eth0]
The example configuration file is an equivalent of the following command's options:
ptp4l -i eth0 -m -S
For a complete list of ptp4l configuration options,
see man 8 ptp4l.
19.2.4 Delay Measurement #
ptp4l measures time delay in two different ways:
peer-to-peer (P2P) or
end-to-end (E2E).
- P2P
This method is specified with
-P.It reacts to changes in the network environment faster and is more accurate in measuring the delay. It is only used in networks where each port exchanges PTP messages with one other port. P2P needs to be supported by all hardware on the communication path.
- E2E
This method is specified with
-E. This is the default.- Automatic method selection
This method is specified with
-A. The automatic option startsptp4lin E2E mode, and changes to P2P mode if a peer delay request is received.
All clocks on a single PTP communication path must use the same method to measure the time delay. A warning will be printed if either a peer delay request is received on a port using the E2E mechanism, or an E2E delay request is received on a port using the P2P mechanism.
19.2.5 PTP Management Client: pmc #
You can use the pmc client to obtain more detailed
information about ptp41. It reads from the standard
input—or from the command line—actions specified by name and
management ID. Then it sends the actions over the selected transport,
and prints any received replies. There are three actions supported:
GET retrieves the specified information,
SET updates the specified information, and
CMD (or COMMAND) initiates the
specified event.
By default, the management commands are addressed to all ports. The
TARGET command can be used to select a particular
clock and port for the subsequent messages. For a complete list of
management IDs, run pmc help.
pmc -u -b 0 'GET TIME_STATUS_NP'
sending: GET TIME_STATUS_NP
90f2ca.fffe.20d7e9-0 seq 0 RESPONSE MANAGMENT TIME_STATUS_NP
master_offset 283
ingress_time 1361569379345936841
cumulativeScaledRateOffset +1.000000000
scaledLastGmPhaseChange 0
gmTimeBaseIndicator 0
lastGmPhaseChange 0x0000'0000000000000000.0000
gmPresent true
gmIdentity 00b058.feef.0b448a
The -b option specifies the boundary hops value in sent
messages. Setting it to zero limits the boundary to the local
ptp4l instance. Increasing the value will retrieve
the messages also from PTP nodes that are further from the local
instance. The returned information may include:
- stepsRemoved
The number of communication nodes to the grandmaster clock.
- offsetFromMaster, master_offset
The last measured offset of the clock from the master clock (nanoseconds).
- meanPathDelay
The estimated delay of the synchronization messages sent from the master clock (nanoseconds).
- gmPresent
If
true, the PTP clock is synchronized to the master clock; the local clock is not the grandmaster clock.- gmIdentity
This is the grandmaster's identity.
For a complete list of pmc command line options, see
man 8 pmc.
19.3 Synchronizing the Clocks with phc2sys #
Use phc2sys to synchronize the system clock to the PTP
hardware clock (PHC) on the network card. The system clock is considered
a slave, while the network card a
master. PHC itself is synchronized with
ptp4l (see Section 19.2, “Using PTP”). Use
-s to specify the master clock by device or network
interface. Use -w to wait until ptp4l
is in a synchronized state.
phc2sys -s eth0 -w
PTP operates in International Atomic Time (TAI),
while the system clock uses Coordinated Universal
Time (UTC). If you do not specify -w to wait
for ptp4l synchronization, you can specify the offset
in seconds between TAI and UTC with -O:
phc2sys -s eth0 -O -35
You can run phc2sys as a service as well:
systemctl start phc2sys
In this case, phc2sys reads its options from the
/etc/sysconfig/phc2sys file. For more information on
phc2sys options, see man 8 phc2sys.
To enable the phc2sys service permanently, run the
following:
systemctl enable phc2sys
To disable it, run
systemctl dosable phc2sys
19.3.1 Verifying Time Synchronization #
When PTP time synchronization is working properly and hardware time
stamping is used, ptp4l and
phc2sys output messages with time offsets and
frequency adjustments periodically to the system log.
An example of the ptp4l output:
ptp4l[351.358]: selected /dev/ptp0 as PTP clock ptp4l[352.361]: port 1: INITIALIZING to LISTENING on INITIALIZE ptp4l[352.361]: port 0: INITIALIZING to LISTENING on INITIALIZE ptp4l[353.210]: port 1: new foreign master 00a069.eefe.0b442d-1 ptp4l[357.214]: selected best master clock 00a069.eefe.0b662d ptp4l[357.214]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE ptp4l[359.224]: master offset 3304 s0 freq +0 path delay 9202 ptp4l[360.224]: master offset 3708 s1 freq -28492 path delay 9202 ptp4l[361.224]: master offset -3145 s2 freq -32637 path delay 9202 ptp4l[361.224]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED ptp4l[362.223]: master offset -145 s2 freq -30580 path delay 9202 ptp4l[363.223]: master offset 1043 s2 freq -28436 path delay 8972 [...] ptp4l[371.235]: master offset 285 s2 freq -28511 path delay 9199 ptp4l[372.235]: master offset -78 s2 freq -28788 path delay 9204
An example of the phc2sys output:
phc2sys[616.617]: Waiting for ptp4l... phc2sys[628.628]: phc offset 66341 s0 freq +0 delay 2729 phc2sys[629.628]: phc offset 64668 s1 freq -37690 delay 2726 [...] phc2sys[646.630]: phc offset -333 s2 freq -37426 delay 2747 phc2sys[646.630]: phc offset 194 s2 freq -36999 delay 2749
ptp4l normally writes messages very frequently. You
can reduce the frequency with the summary_interval
directive. Its value is an exponent of the 2^N expression. For example,
to reduce the output to every 1024 (which is equal to 2^10) seconds, add
the following line to the /etc/ptp4l.conf file:
summary_interval 10
You can also reduce the frequency of the phc2sys
command's updates with the -u
SUMMARY-UPDATES option.
19.4 Examples of Configurations #
This section includes several examples of ptp4l
configuration. The examples are not full configuration files but rather
a minimal list of changes to be made to the specific files. The string
ethX stands for the actual network interface
name in your setup.
/etc/sysconfig/ptp4l:
OPTIONS=”-f /etc/ptp4l.conf -i ethX”
No changes made to the distribution
/etc/ptp4l.conf.
/etc/sysconfig/ptp4l:
OPTIONS=”-f /etc/ptp4l.conf -i ethX”
/etc/sysconfig/phc2sys:
OPTIONS=”-s ethX -w”
No changes made to the distribution
/etc/ptp4l.conf.
/etc/sysconfig/ptp4l:
OPTIONS=”-f /etc/ptp4l.conf -i ethX”
/etc/sysconfig/phc2sys:
OPTIONS=”-s CLOCK_REALTIME -c ethX -w”
/etc/ptp4l.conf:
priority1 127
/etc/sysconfig/ptp4l:
OPTIONS=”-f /etc/ptp4l.conf -i ethX”
/etc/ptp4l.conf:
priority1 127
19.5 PTP and NTP #
NTP and PTP time synchronization tools can coexist, synchronizing time from one to another in both directions.
19.5.1 NTP to PTP Synchronization #
When ntpd is used to synchronize the local
system clock, you can configure the ptp4l to be the
grandmaster clock distributing the time from the local system clock via
PTP. Include the priority1 option in
/etc/ptp4l.conf:
[global] priority1 127 [eth0]
Then run ptp4l:
ptp4l -f /etc/ptp4l.conf
When hardware time stamping is used, you need to synchronize the PTP
hardware clock to the system clock with phc2sys:
phc2sys -c eth0 -s CLOCK_REALTIME -w
19.5.2 PTP to NTP Synchronization #
You can configure ntpd to distribute the time
from the system clock synchronized by ptp4l or
phc2sys by using the local
reference clock driver. Moreover, you need to stop
ntpd from adjusting the system clock—do
not specify any remote NTP servers in
/etc/ntp.conf:
server 127.127.1.0 fudge 127.127.1.0 stratum 0
When the DHCP client command dhclient receives a
list of NTP servers, it adds them to NTP configuration by default. To
prevent this behavior, set
NETCONFIG_NTP_POLICY=""
in the /etc/sysconfig/network/config file.