While viewing this page see also:

"right" tz database (zoneinfo) files and GPS-based NTP

How a POSIX-based system can have the right number of elapsed seconds, the right time, and never require the kernel to implement a leap second

A version of this web page was presented at the October 2011 meeting Decoupling Civil Timekeeping from Earth Rotation. Here are PDF files of the slides and preprint.

Existing operational and tested system interfaces and code could continue to keep leap seconds in UTC even if the ITU-R removes them from the internationally-approved broadcast time scale

There are so many existing legal and technical requirements which specify UTC that nobody is going to consider any action which would result in the nonexistence of a time scale internationally recognized as UTC.

Therefore, any change of name of the broadcast time scale can only be considered if there already exists a mechanism by which the radio time scale can have a new name while it remains possible to also produce a time scale named UTC.

The point of this web page is to demonstrate that such a mechanism exists and is broadly deployed. Furthermore, it is supported by every vendor who wants their machines to track all the politicians and bureaucrats who keep on changing the timezone offsets.

As of today we are at tzdata2011h, which means that every vendor has already issued 8 changes of the database during this calendar year. Those changes result in offsets of an hour, not a second, and they are far less predictable than leap seconds.

Skip all this verbiage and jump to the worked example.
Alternatively, here's the description of how to do it.

The NTP server

The NTP clients

This results in NTP client machines which are running their POSIX system clock values of time_t based on GPS time. GPS time never has leap seconds, so the operating system of such machines does not have the complex issues in the kernel that happen when leap seconds reset the clock. These systems also produces a correctly-formatted local date/time string for every "right" time zone.

Why does this work?

The "right" files in the tz (zoneinfo) database have a subtle difference from the POSIX standard. POSIX requires that the system clock value of time_t represent the number of non-leap seconds since 1970-01-01. This is the same as requiring POSIX seconds to be mean solar seconds of UT, not the atomic seconds that UTC has counted since 1972-01-01.

The "right" zoneinfo files assert that the system clock value of time_t represent the actual number of seconds in the internationally approved broadcast time scale since 1970-01-01. As a result the value of time_t which is expected by the "right" zoneinfo files is greater than the value of time_t specified by POSIX. The difference in the values of time_t is the number of leap seconds which have been inserted into the internationally approved broadcast time scale. As of year 2011 the difference is 24 seconds.

The scheme described in this web page uses a non-standard NTP server and a non-standard set of "right" zoneinfo files. The hard part is that the zoneinfo files must be hacked for GPS time and recompiled whenever a leap second is announced. Hopefully that recompile happens long before the leap second occurs. In this scheme the kernel does not have to handle the leap second. All of the handling of the leap second happens in the zoneinfo files. This is effectively the same as the bi-annual handling of daylight/summer time transitions. There are no real-time changes. Everything about the changes can be easily tested at any time by any user.

How is this related to the ITU-R action to eliminate leap seconds?

The implementation of leap seconds in the internationally broadcast time scale is described by ITU-R Recommendation TF.460-6 (the 6th revision of the document originally created in 1970 as CCIR Rec. 460). Early in the year 2012 the Radiocommunications Assembly of the ITU-R will probably vote on a draft revision TF.460-6. The current text of that draft revision calls for radio broadcasts of time signals to abandon leap seconds 5 years after it is approved.

During the process of drafting the revision the SRG of WP7A of the ITU-R held a colloquium of international timekeeping experts in Torino Italy on the future of UTC during 2003 May (link1, link2). The conclusion (link1, link2) reached at that colloquium said that if leap seconds are removed from the internationally approved broadcast time scale then the name of that time scale should be changed. The experts suggested that a new name for the broadcast time scale might be TI (International Time).

The current draft TF.460-6 keeps the name UTC for the broadcast time scale. This disregards the advice given to the ITU-R by the experts at the Torino colloquium.

Many of the difficulties with leap seconds arise in computing systems. At no point did the ITU-R engage the IETF, the IANA, the NTP working group nor any other computing group in discussions of the possible options for changing broadcast time signals. The scheme described in this web page has never been seriously considered in an official forum.

Unlike the current draft revision TF.460-6, the scheme described here is truly a compromise. It removes the existing problem of leaps in the broadcast time scale. It acknowledges that technological civilization now depends on atomic timekeeping while preserving the traditional meaning of the word "day". It points the way for national legislation to recognize both kinds of time without breaking existing precedent. But it can only work if the ITU-R changes the name of the broadcast time scale along with removing leap seconds from the broadcast time scale.

What does POSIX actually require?

The POSIX standard requires that the value of time_t have a simple relationship to date and time where each day has 86400 seconds. Such a requirement demands that POSIX systems not count leap seconds because they produce days which have 86401 seconds.

The POSIX standard also specifies that the time and date values be UTC. UTC has had leap seconds, so the combination of the two requirements is inconsistent with the broadcast time signals that are used to set system clocks.

This is self-inconsistent definition is made more problematic because other specifications require POSIX compliance. One of these is the US US Federal Information Processing Standard ( FIPS PUB 151-2)

There is no way to change POSIX away from the simple relationship of 86400 seconds in a day; there are too many existing applications which presume this to be the case. It is possible to change the meaning of "POSIX day" to be "atomic day" of 86400 seconds of TAI.

There is no way to change POSIX away from setting the system clock value of time_t to the value of the existing broadcast time signals. This way of setting the clock is the scheme which has evolved and been implemented by too many existing systems. (The scheme of using GPS time described in this web page is a hack which will only ever be used by special-purpose systems.)

The operational matters of systems are different than the words of the POSIX standard. It is true that the only option for setting the system clocks of computers is the internationally approved broadcast time scale. At present that broadcast time scale is known to humans as UTC, but the machines are oblivious to that name. No matter what the broadcast time scale is called, that is what is going to be used by the vast majority of machines.

POSIX already requires that the zoneinfo mechanism be able to handle offsets expressed in hours, minutes, and seconds between time_t and the local timezone (controlled by environment variable TZ). That means the existing implementations can allow time_t to be a broadcast time scale called something other than UTC, and the non-broadcast civil UTC could become a time zone just like all the other civil time zones which are offset from time_t.

Variations on this scheme

A system whose time_t is set according to POSIX (i.e., with an NTP server giving UTC, thus the kernel must handle leap seconds) and which has been configured to use the unhacked "right" zoneinfo files will produce formatted date/time strings which are 24 seconds smaller than official time. (The value -24 will remain constant at each leap second.)
This is a condition which does arise when system managers or users mistakenly choose "right" zones.

If the NTP server conforms to UTC then a system which wants to produce formatted date/time strings that match official time and which wants to use the "right" zoneinfo files as distributed must set its time_t 24 seconds larger than a POSIX system would have.
This would require a hacked version of the NTP client which adds 24 seconds to time_t. The hard part of is that the value 24 must be incremented in real time every time there is a leap second. Worst of all, the kernel must still handle the leap second. Therefore this variation requires two different subsystems to perform hard-to-test changes in real time. So this variation is even worse than the POSIX standard requires.

A system whose time_t is set using an NTP server giving GPS time (thus the kernel does not have to handle leap seconds) and which has been configured to use the unhacked "right" zoneinfo files will produce formatted date/time strings which are 9 seconds smaller than official time. (The value -9 will remain constant at each leap second.)

A system whose time_t is set using an NTP server giving GPS time (thus the kernel does not have to handle leap seconds) and which is configured to use the usual zoneinfo files will produce formatted date/time strings which are 15 seconds larger than official time. (The value 15 will increment at each leap second.)
According to the developer forums this is the variation that Google has chosen for Android devices.

Some NTP servers are capable of producing TAI instead of UTC. In that case the "right" zoneinfo files could be hacked to insert 10 fictitious leap seconds (probably in the early 1970s). This would also result in systems which never have to handle leap seconds in the kernel and which produce correct time in the formatted date/time strings.

Objections

I first described the scheme on this web page on the LEAPSECS e-mail list in 2009. Further discussion occurred in 2011. A few of the concerns in the replies are noted below along with my response.

updating zoneinfo files is difficult, sometimes impossible
cold-spare systems need 5 or 10 years of forewarning
Systems which cannot update zoneinfo files also cannot be aware of changes in daylight/summer time transitions. Politicians in charge of timezones have often given even less notice of a change than the IERS gives before leap seconds.
In such cases the system is not capable of giving a correct local time, so its only option is to give the international broadcast time scale which is being used to set time_t. If the leap seconds are removed from the broadcasts and put into the zoneinfo files, and if the advice from Torino is followed, then the name of the time scale which is predictable would be TI instead of UTC.

each installed version of java has its own set of zoneinfo files
This means that different versions of java on the same machine could give civil times that differ by an hour. Leap seconds are a small issue by comparison.

some programs assume that each day has 86400 seconds when they calculate the time_t of an event at the same time on subsequent days
Programs like these are already broken in many countries where every year there is a day that is 23 hours long and a day that is 25 hours long.

long-running daemons only consult zoneinfo once at startup, so they would not receive notice of new leap second announcements
Such programs are usually doing their timestamping in local civil time, and in some jurisdictions the daylight/summer offsets change with less warning than leap seconds, so (as above) programs like this are already broken. Furthermore, any daemon currently producing timestamps in UTC should find it equally valid to produce timestamps in TI, and that would be unaffected by daylight and leaps.

some programs implement their own version of the conversion between time_t and formatted date/time rather than relying on the ones provided by the operating system
There will be 5 years to get ready for a change, and after the change the difference in times will remain only a few seconds for many years.

the time_t value of a future civil event would not be predictable if leap seconds continue
Due to ongoing changes of daylight/summer transition dates this is already the case by an hour in many countries, and the abandonment of leap seconds cannot fix the politicians. Specification of future dates also requires specification of the intended time scale in order that the actual time can be computed unambiguously.

the establishment of a new time scale TI in addition to keeping UTC would be confusing
Humans can learn more easily than machines and their protocols. Nevertheless, many existing humans, textbooks, and software presume that UTC is mean solar time and it will take a generation before that is remedied.
The scheme described here favors the continuity of operational procedures for the machines without destroying the traditional meaning of the word "day".

the establishment of a new time scale TI would want a single-letter military-style timezone abbreviation
Given that the GPS time scale does not have such an abbreviation it is not clear why TI would need one. All of the English alphabet 'A' through 'Z' is in use except for 'J', so 'J' could be used. Alternatively, the character '@' could be used with the obvious mnemonic "atomic".

this POSIX scheme does not apply to Microsoft operating systems
Microsoft has rejected the notion that Windows is capable of noticing leap seconds or their absence because
The W32Time service cannot reliably maintain sync time to the range of 1 to 2 seconds. Such tolerances are outside the design specification of the W32Time service.
If the ITU-R gives them 5 years of notice then I have faith that Microsoft will attend to the issue.

Caveats

Hacked NTP servers which produce a time scale other than UTC could be a source of confusion for conforming NTP clients which expect UTC. Such NTP servers are frowned upon and must be very carefully managed so that they are not mistakenly used by normal NTP clients.

Because POSIX has required that systems not acknowledge the existence of leap seconds the values of time_t do not conform to UTC when there are leap seconds. As a result there are values of time_t which are ambiguous. This flaw cannot ever be remedied by any changes to any standards or protocols. The past is broken, all we can do is fix the future.

Systems using the scheme on this web page will not satisfy the POSIX requirement for the correspondence between values of time_t and formatted date/time strings.

Systems using the scheme on this web page are not POSIX-compliant and MUST NOT exchange a time_t value with any POSIX-compliant system.

Various utilities for exchanging files between systems such as ftp, rsync, NFS, etc. rely on time_t or the formatted date/time string, or both. Prior experience with the "right" zoneinfo files has already resulted in strange behaviors. Although these strange behaviors mean that no attempt may be made to fix the past, it remains possible to fix the future.

In a heterogeneous computing environment the notion of hacking the zoneinfo files and/or using the "right" zoneinfo files is a system management nightmare. This would only be likely to be used in a closed environment where the operational systems must avoid leap seconds yet also must match official time in formatted date/time strings. Whereas the scheme described on this page demonstrates that it is possible to retain mean solar time while not having leap seconds in the broadcasts, the only feasible way to get universal vendor cooperation would be for the ITU-R to change the name of the broadcast time scale (as the new underlying basis for POSIX time) and let UTC become the responsibility of the IERS (as a new variety of time zone).

The roles of the ITU-R, IAU, BIPM, IERS

The ITU-R is an international regulatory body. It produces recommendations which are proprietary documents that may not be openly distributed. The ITU-R is not required to describe implementation details nor to demonstrate interoperability before they adopt their recommendations. Implementation and interoperability are Somebody Else's Problem. The ITU-R structure is not conducive to open discussions of the details of leap seconds.

In 1970 the CCIR (predecessor to the ITU-R) wrote Recommendation 460 with no description of implementation. All implementation details for leap seconds were produced by the IAU and the time service bureaus. The task of determining leap seconds was given to the BIH. In 1988 the BIH ceased to exist, and in 1993 so did the CCIR.

The BIPM implements the SI, the international system of units. Since 1988 one of those implementations is TAI, but that is not available for operational systems because its value is not known until it is published a month after the fact. The BIPM rejects responsibility for civil time scales.

Since 1988 the Paris bureau of the IERS has responsibility for announcing leap seconds. The IERS does not have the means for distributing that information in a robust fashion for modern operational systems. The most robust scheme for distributing upcoming leap seconds is implemented by the GPS satellites. The most robust scheme for distributing all leap seconds is the leapseconds file in the tzdata/tzcode that make up the IANA Time Zone (tz/zoneinfo) distribution.

In my opinion the existing ensemble of agencies who are responsible for UTC is a dysfunctional family. The result is a chaotic mess of contradictory, self-inconsistent, and untestable standards and implementations. As a result almost every discussion on the subject turns into a flame war. To their credit, these agencies have been streamlining the mess of recommendations and statements during the past few years.

Nevertheless, operational systems which must avoid leap seconds have little option other than to pick which requirements they are going to disregard. One way out of this dysfunctional mess is for the ITU-R to get out of the business of describing the basis of the civil time scale by declaring that their broadcast time scale is intended only for technical purposes.


Example

This runs on an old Linux box.
It also works on Mac OS X 10.6

the hacked leapseconds file

# Leap  YEAR    MONTH   DAY     HH:MM:SS        CORR    R/S
Leap    1981    Jun     30      23:59:60        +       S
Leap    1982    Jun     30      23:59:60        +       S
Leap    1983    Jun     30      23:59:60        +       S
Leap    1985    Jun     30      23:59:60        +       S
Leap    1987    Dec     31      23:59:60        +       S
Leap    1989    Dec     31      23:59:60        +       S
Leap    1990    Dec     31      23:59:60        +       S
Leap    1992    Jun     30      23:59:60        +       S
Leap    1993    Jun     30      23:59:60        +       S
Leap    1994    Jun     30      23:59:60        +       S
Leap    1995    Dec     31      23:59:60        +       S
Leap    1997    Jun     30      23:59:60        +       S
Leap    1998    Dec     31      23:59:60        +       S
Leap    2005    Dec     31      23:59:60        +       S
Leap    2008    Dec     31      23:59:60        +       S
Leap    2011    Jul     15      17:30:60        +       S
That last entry is bogus for the sake of this test.

the script

Note that this uses tzdata2011h, already the 8th release of the tz database during year 2011. The politicians and bureaucrats in charge of civil time issue new rules far more often and with much less notice than leap seconds. The system vendors of the world have established robust means for distributing new zoneinfo files. Anyone who does not update the zoneinfo risks being off by a full hour when participating in international telecommunication.
#! /bin/sh

then='empty'
while true; do
  now=`date`
  if [ x"$now" != x"$then" ]; then
    right=`TZ=:$HOME/zoneinfo2011h+gps/etc/zoneinfo-leaps/US/Pacific date`
    time_t=`date +%s`
    echo "$time_t POSIX $now right+GPS $right"
    then=$now
  fi
  usleep 50000
done

the output

Note on the right column the timestamp 10:30:60 along with the change from 15 seconds offset to 16 seconds offset, but the time_t value increments uniformly with no leap.
1310751055 POSIX Fri Jul 15 10:30:55 PDT 2011 right+GPS Fri Jul 15 10:30:40 PDT 2011
1310751056 POSIX Fri Jul 15 10:30:56 PDT 2011 right+GPS Fri Jul 15 10:30:41 PDT 2011
1310751057 POSIX Fri Jul 15 10:30:57 PDT 2011 right+GPS Fri Jul 15 10:30:42 PDT 2011
1310751058 POSIX Fri Jul 15 10:30:58 PDT 2011 right+GPS Fri Jul 15 10:30:43 PDT 2011
1310751059 POSIX Fri Jul 15 10:30:59 PDT 2011 right+GPS Fri Jul 15 10:30:44 PDT 2011
1310751060 POSIX Fri Jul 15 10:31:00 PDT 2011 right+GPS Fri Jul 15 10:30:45 PDT 2011
1310751061 POSIX Fri Jul 15 10:31:01 PDT 2011 right+GPS Fri Jul 15 10:30:46 PDT 2011
1310751062 POSIX Fri Jul 15 10:31:02 PDT 2011 right+GPS Fri Jul 15 10:30:47 PDT 2011
1310751063 POSIX Fri Jul 15 10:31:03 PDT 2011 right+GPS Fri Jul 15 10:30:48 PDT 2011
1310751064 POSIX Fri Jul 15 10:31:04 PDT 2011 right+GPS Fri Jul 15 10:30:49 PDT 2011
1310751065 POSIX Fri Jul 15 10:31:05 PDT 2011 right+GPS Fri Jul 15 10:30:50 PDT 2011
1310751066 POSIX Fri Jul 15 10:31:06 PDT 2011 right+GPS Fri Jul 15 10:30:51 PDT 2011
1310751067 POSIX Fri Jul 15 10:31:07 PDT 2011 right+GPS Fri Jul 15 10:30:52 PDT 2011
1310751068 POSIX Fri Jul 15 10:31:08 PDT 2011 right+GPS Fri Jul 15 10:30:53 PDT 2011
1310751069 POSIX Fri Jul 15 10:31:09 PDT 2011 right+GPS Fri Jul 15 10:30:54 PDT 2011
1310751070 POSIX Fri Jul 15 10:31:10 PDT 2011 right+GPS Fri Jul 15 10:30:55 PDT 2011
1310751071 POSIX Fri Jul 15 10:31:11 PDT 2011 right+GPS Fri Jul 15 10:30:56 PDT 2011
1310751072 POSIX Fri Jul 15 10:31:12 PDT 2011 right+GPS Fri Jul 15 10:30:57 PDT 2011
1310751073 POSIX Fri Jul 15 10:31:13 PDT 2011 right+GPS Fri Jul 15 10:30:58 PDT 2011
1310751074 POSIX Fri Jul 15 10:31:14 PDT 2011 right+GPS Fri Jul 15 10:30:59 PDT 2011
1310751075 POSIX Fri Jul 15 10:31:15 PDT 2011 right+GPS Fri Jul 15 10:30:60 PDT 2011  leap second
1310751076 POSIX Fri Jul 15 10:31:16 PDT 2011 right+GPS Fri Jul 15 10:31:00 PDT 2011
1310751077 POSIX Fri Jul 15 10:31:17 PDT 2011 right+GPS Fri Jul 15 10:31:01 PDT 2011
1310751078 POSIX Fri Jul 15 10:31:18 PDT 2011 right+GPS Fri Jul 15 10:31:02 PDT 2011
1310751079 POSIX Fri Jul 15 10:31:19 PDT 2011 right+GPS Fri Jul 15 10:31:03 PDT 2011
1310751080 POSIX Fri Jul 15 10:31:20 PDT 2011 right+GPS Fri Jul 15 10:31:04 PDT 2011
1310751081 POSIX Fri Jul 15 10:31:21 PDT 2011 right+GPS Fri Jul 15 10:31:05 PDT 2011
1310751082 POSIX Fri Jul 15 10:31:22 PDT 2011 right+GPS Fri Jul 15 10:31:06 PDT 2011
1310751083 POSIX Fri Jul 15 10:31:23 PDT 2011 right+GPS Fri Jul 15 10:31:07 PDT 2011
1310751084 POSIX Fri Jul 15 10:31:24 PDT 2011 right+GPS Fri Jul 15 10:31:08 PDT 2011
1310751085 POSIX Fri Jul 15 10:31:25 PDT 2011 right+GPS Fri Jul 15 10:31:09 PDT 2011
1310751086 POSIX Fri Jul 15 10:31:26 PDT 2011 right+GPS Fri Jul 15 10:31:10 PDT 2011
1310751087 POSIX Fri Jul 15 10:31:27 PDT 2011 right+GPS Fri Jul 15 10:31:11 PDT 2011
1310751088 POSIX Fri Jul 15 10:31:28 PDT 2011 right+GPS Fri Jul 15 10:31:12 PDT 2011
1310751089 POSIX Fri Jul 15 10:31:29 PDT 2011 right+GPS Fri Jul 15 10:31:13 PDT 2011
1310751090 POSIX Fri Jul 15 10:31:30 PDT 2011 right+GPS Fri Jul 15 10:31:14 PDT 2011

If the ITU-R were to change the name of the broadcast time scale along with stopping leap seconds then the two columns here could be named "TI" (the underlying uniform time of the broadcasts) and "UTC" (the new master civil time zone from which all other time zones would be offset). This is what the international experts recommended to the ITU-R at the end of the 2003 colloquium in Torino.

As seen here this scheme just works, yet the ITU-R has never engaged in discussions regarding its applicability.

Subsequent events

This scheme to allow a POSIX kernel to avoid handling leap seconds was presented at the 2011 October Future of UTC meeting in Exton. During 2012 January the Radiocommunication Assembly of the ITU-R declined to vote on the draft proposed change to TF.460-6 which would have redefined the word "day" such that it would become unrelated to the rotation of the earth. Immediately afterward the World Radio Conference of the ITU-R directed Working Party 7A "to ensure that all the technical options have been fully addressed" prior to the Radiocommunication Assembly scheduled for 2015. The likelihood is that any change recommended in 2015 cannot be implemented in less than 5 years after that meeting.

As seen during the leap second of 2012-06-30, the system managers of the world cannot wait for the ITU-R to act. Amazon.com had no significant issues attributed to the leap, apparently because their sysadmins had already analyzed the vulnerabilities and taken appropriate precautions. Google.com had already revealed their leap smear where they run an internal non-standard NTP service that adjusts the duration of seconds to avoid the leap. Various distribution in the Linux world have been switching from ntpd to chrony because chrony prefers to adjust the duration of seconds rather than to step (or leap) the system clock.

The "smear" strategy works well for cloud-based, virtualized computing which is not performing real-time tracking or control. It is probably not fair to say that the Site Reliability team at Google and the sysadmins who are switching from ntpd to chrony are rejecting the 1970 CCIR decision that changed civil time keeping from "rubber seconds" to leap seconds. More likely these admins are simply taking the most expedient steps to ensure that their systems have no problems. It is much less clear what strategies are being adopted by admins with systems that do perform real-time actions; those sysadmins are not publicly releasing information about how they handle leap seconds.


Steve Allen <sla@ucolick.org>
$Id: right+gps.html,v 1.60 2014/06/06 04:00:58 sla Exp $