The Trouble with POSIX

From: Peter Bunclark <psb_at_ast.cam.ac.uk>
Date: Fri, 30 Sep 2005 15:10:14 +0100 (BST)

I present below a distillation of many of the comments which mention
POSIX from the leapsecs mailing list. I apologise unashamedly for my
cuts and selections, and apologise profusely on the off chance I got
the attributions wrong.

I started doing this for my own use, but then thought perhaps some of you
might be interested in seeing the material again in summary form.

It's interesting that some people quote the POSIX standard to support their
argument about leap seconds, yet have variant opinions. Perhaps this is
telling us that some of us find the POSIX standard a bit too hard to
understand, or, perhaps, that it could be better written.

Also, there does seem to be a tendency for people to think that the problems
with timekeeping are best solved by others changing their ways, after
all, I'm already perfect.

Now that we can synchronize clocks globally and beyond, we see that there
are problems all over the shop. A global solution is required, not just
a fix to somebody else's bit of software, but which considers civil-legal
time, scientific time, POSIX-like standards, ntp-like time distribution and
implementation details.

You wouldn't think a second here or there would matter that much...

Peter.

-----------------------Selected Quotes from LeapSecs:------------------------
Markus Kuhn
        considers:
g) that numerous information and communication systems use an internal
   time scale based on a fixed length of the day of 86400 seconds, in
   which there exists no unique representation for points in time
   during an inserted UTC leap second, including the widely used POSIX
   time scale defined by ISO/IEC 9945-1:1996 in section 2.2.2.113,

Garrett Wollman
        notes:
The requirement that I've heard most commonly is much simpler: there
must be 86,400 nominal seconds per nominal day, and nominal days must
be the same ordinal and duration as provided by local law and custom.
The POSIX specification makes the former requirement explicit, by
giving a formula purporting to relate ``seconds since the epoch'' to a
civil date and time (not accounting for time zones).

Paul Eggert
        points out:
the UTC markers are 23:59:59, 23:59:60, 00:00:00. Second -- and this
is a more subtle point -- UTC is set back immediately after an
inserted leap second, which means that if your clock has 86,400
seconds per day (as is required for POSIX applications, for example),
then it should tick 23:59:59, 00:00:00, 00:00:00. The POSIX clock is
not set back to 23:59:59; it is set back to 00:00:00.
It is this sort of confusion (even among experts!) that causes many
people to think that there must be a better way to handle civil time,
a way that does not involve discontinuities.

Michael Deckers
        responded:
  What you describe may be required by POSIX but it is wrong for UTC: the
  second starting with the marker 23:59:60 is called "leap second" in UTC
  and (more importantly) it belongs to June or December, not to July or
  January. I quote from [ITU-R Rec. 460-4, section 2.2]:
      "A positive leap-second begins at 23h 59m 60s and ends
       at 0h 0m 0s of the first day of the following month."

Markus Kuhn
        adds
What *does* literally get set back indeed is the POSIX clock and similar
representations, which use by definition the same numeric second counter
value to represent (day D) 23:59:60.xxx and (day D+1) 00:00:00.xxx. That
is obviously unpleasant, as it leads to non-causal timestamps and it is
easy to construct scenarios where this could mess up things in theory.

        and, plugging UTS:
It's merely a common missunderstanding of the
definition of POSIX timestamps. There exists already a perfectly simple
algorithmic leap-second-table-free mapping between Unix-style timestamps
and UTC, specified formally in

  ISO/IEC 9945-1:1996, Section 2.2.2.113
  http://www.cl.cam.ac.uk/~mgk25/volatile/posix-2-2-2-113.pdf

Unix timestamps have always been meant to be an encoding of a
best-effort approximation of UTC. They have always counted the non-leap
seconds since 1970-01-01. The only minor problem is that the value
23:59:60 cannot be represented uniquely in the time_t encoding, but that
is in practice elegantly circumvented by changing the length of the Unix
second near a UTC leap second by less than a percent (UTC smoothing,
something which I suggest should be standardized formally for Unix-style
timestamps to improve interoperability of timestamps near leap seconds).
The older POSIX.1:1996 interpretation above could be quoted as implying
that time_t has to jump back during a leap second because the formula
provided leads to the same numeric value for 23:59:60 and 00:00:00 the
next day (unfortunately, that is still what the Linux NTP kernel support
does today). The POSIX.1:2001 revision softened the definition in order
to include the option of UTC smoothing into what it allows, making it
possible to use a more graceful leap second representation in time_t,
such as for example my UTS proposal.

Glen Seeds writes at length:
The POSIX timestamping community has already had many long discussions of
these issues. The current state of the standard reflects several realities.

The first reality is that POSIX implementors and application writers, who
are the only targets audiences of the POSIX standards, have no control over
whether time_t is TAI, UTC, or anything else. This is under the control of
the system owners and operators, and the host environment that the POSIX
implementation is embedded in.

The second reality is that many existing applications depend on calculations
that assume that time_t has exactly 86400 seconds per year. (Note that it
does NOT follow from this that there are 86400 POSIX seconds in any given
calendar year. It does follow that if there are more than that, the
remainder are hard to represent in POSIX.)

The third reality is that almost all POSIX systems require a time
representation that is easily relatable to local time. UTC+TZ is the only
readily available precise measure of local time, so mandating TAI would
conflict with this. (For this reason, the bias is the POSIX world is heavily
toward UTC rather than TAI, and is behind much of the resistance to dropping
leap seconds from UTC)

POSIX Real Time does recognize and mandate that there must be a
monotonically increasing clock. It cannot and does not mandate that ALL
sources of time_t are monotonic.

There has been discussion in this forum of "smoothed" leap seconds. Much of
this was born of discussions within the POSIX community some years ago, and
many existing implementations provide for smoothed time adjustment that
guarantees a non-decreasing clock.

ut1-mail_at_ASTHE.COM
        ripostes somewhat anonymously:
Nothing in POSIX requires time (local, time_t, etc.) to be precise.
.. the time_t can be anything ... including TAI, UTC, a bogus clock, etc.
    * So time_t can be related to almost anything.
    * The POSIX "TZ=UTC" names computed from those seconds since the Epoch"
      time_t timestamp values can therefore related to almost anything.
    * Timezones such as "PST8PDT" are 8 hours off of POSIX "TZ=UTC",
      can therefore related to almost anything.
And thus the claim of "mandating TAI would conflict with this" appears
to be false.

Steve Allen
        says
I stand by the tense of my statement: time_t *should* be TAI in order
that POSIX timestamps can be unambiguous and monotonically increasing.
I fully recognize the realities that prevent it from being TAI. And I
applaud the POSIX Real Time folks for taking the broader view which,
when fully implemented, will permit compatibility with existing usage
as well as monotonicity.

Glen Seeds
        had the opinion
I believe you would get very strong pushback from the POSIX world if you
tried to make the time base TAI rather than UTC. Without a strong case
for this change (which I haven't seen yet), I would be one of them.

Steve Allen
        again says
Even the POSIX standard and the majority of Unix systems that obey it
are self-inconsistent in the presence of leap seconds. In a sentence
the standard says that POSIX time should be taken to be UTC, except
when it should not.

Glen Seeds
        reminds us
The POSIX clock is explicitly and deliberately not aligned with UTC or
any other absolute time standard. Facilities for converting the system
clock to local time are implementation-specific.

Mark Calabretta
        lets us know
Apparently such "complexity in the library functions" was what swayed
the POSIX committee against using TAI in the first place. On the bright
side, the wikipedia URL contains the following statement:

   As of 2004, POSIX has new interfaces making several different time
   scales available to programs, splitting up the many uses to which
   Unix times have traditionally been put. The future is one where time
   values are accompanied by explicit labels of the time scale defining
   their significance. Unix time as described in this article will
   still be in wide use for decades to come, but is likely to be
   increasingly treated as a legacy system and superseded by better-
   defined systems.

An optimistic interpretation of this would be that POSIX.1 will provide
for unix time to be maintained as TAI.

A pessimistic scenario would be that finally when POSIX caught up with
leap seconds they were dropped!

Poul-Henning Kamp
        has:
come to the conclusion that while POSIX is a good thing, we shouldn't
let it get in the way of a correct solution.

I've never been able to nail down who made this particular POSIX_MISTAKE
btw, a post-factum interview could be interesting :-)

Poul-Henning Kamp
        digs in:
Read my lips: No rubber seconds.
Yes, it is about as bad as UTS, but bad in different ways. It cannot
be fixed as long as we stick with the POSIX_MISTAKE and still have
leap seconds.

Poul-Henning Kamp
        with the Answer:
Right, and anything POSIX compliant implements this broken.
The fact that POSIX is "fixed" by abandonning leapseconds counts
heavily on the credit side of the cost-benefit of the proposal
because that fixes UNIX, MVS and Windows with one stroke of the
pen.
If on the other hand leapseconds are retained, we need to fix POSIX,
rather than keeping adding weird bandages over the mistake.

Poul-Henning Kamp
        further opines:
The computer guys, even though being too late to the main party
managed to screw things up even more by totally ignoring leap seconds
in what was mindlessly turned into the POSIX standard which is now
a holy text which cannot be changed until the second coming of
Dennis, Ken & Brian.

Steve Allen
        defends:
The POSIX time_t corresponds quite well to the way that civil time has
been maintained throughout history. I doubt that POSIX could have
done anything with time_t other than to supplement it with an
interface which was designed with uniform time in mind.

Markus Kuhn
        also defends:
Until you manage to compile a convincing list of practical and realistic
failure patterns and reports that are *only* caused by the choice that
the POSIX world made with regard to handling leap seconds, I will
continue to consider your opinion to be quite a bit at the far end of
the scale.
I believe the vast majority of POSIX users are very happy with how it
encodes time. The only thing we slightly worry about is that existing
ABIs may keep time_t restricted to 32 bits until 2038.

Poul-Henning Kamp
        suggests:
Deal, you tackle POSIX, I tackle the reference implementations and
NTP.

Poul-Henning Kamp
        continues:
The POSIX definition makes it impossible to correctly handle leap
seconds with any complying implementation of the standard, and
therefore applications which needs to be *truly* leapsecond compliant,
cannot use the standard libraries.

Steve Allen
        asks:
If POSIX were to fix the definitions of time_t and epoch, why would
this not imply that handling leap seconds with Unix would become
trivial?

M. Warner Losh
        replies:
Because leap seconds are not trivial. If you make time_t a TAI-like
thing, then you break all programs that do math to calculate a date
and time since the usual "naive math" no longer works. You could fix
these programs to use new APIs to do the math instead. However, you
also enforce upon all systems a requirement to keep a leap second
table up to date. While not so bad for systems that are constantly
running, this can cause problems for people that maintain a stockpile
of spare parts. These spares generally aren't on when leap updates
occur and won't have them for some period of time after being
deployed. Also, many systems just aren't connected to a public
network, or aren't connected to a network at all, but still have a
need to have knowledge of leap seconds.

John Cowan
        adds:
Because there is far too much code that believes, for example, that if
you add 86400 to a time_t representing 2005-12-31T00:00UTC, you get a
time_t representing 2006-01-31T00:00UTC. Or that if you have a
time difference in seconds, you can divide by 3600 and get one represented
in hours.

The upside of Posix is that time arithmetic is simple. The downside is
that Posix sometimes lies about elapsed time and labels both a leap second
and the preceding second with the same time_t.
Received on Fri Sep 30 2005 - 07:11:47 PDT

This archive was generated by hypermail 2.3.0 : Sat Sep 04 2010 - 09:44:54 PDT