Why only CLOCK_REALTIME time-outs for POSIX semaphores and mutexes?

[Noob]

Hello,

According to POSIX, sem_timedwait and pthread_mutex_timedlock
have time-outs specified in absolute CLOCK_REALTIME unit.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_timedwait.html
The timeout shall be based on the CLOCK_REALTIME clock.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_timedlock.html
The timeout shall be based on the CLOCK_REALTIME clock.

I write software for set-top boxes. When they are powered-on,
they have no idea what time it is. They acquire the current
time and date over a satellite link. By the time we're ready
to read from the satellite tuner, we're pretty far along in
the board's init.

Several software components use semaphores and mutexes, some with
a time-out value. At some point, we acquire the correct time and
date, and we set the system time -- BOOM. Every time-out fires at
the same time.

Question #1 : why didn't POSIX specify allow the possibility to
give CLOCK_MONOTONIC references?

> CLOCK_MONOTONIC
> The identifier for the system-wide monotonic clock, which is
> defined as a clock measuring real time, whose value cannot be set
> via clock_settime() and which cannot have negative clock jumps.
> The maximum possible clock jump shall be implementation-defined.

Question #2 : so how does one deal with the problem of using
clock_settime, while several CLOCK_REALTIME-based time-outs
are expected?

http://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_settime.html

> If the value of the CLOCK_REALTIME clock is set via clock_settime(),
> the new value of the clock shall be used to determine the time of
> expiration for absolute time services based upon the CLOCK_REALTIME
> clock. This applies to the time at which armed absolute timers
> expire. If the absolute time requested at the invocation of such a
> time service is before the new value of the clock, the time service
> shall expire immediately as if the clock had reached the requested
> time normally.
>
> Setting the value of the CLOCK_REALTIME clock via clock_settime()
> shall have no effect on threads that are blocked waiting for a
> relative time service based upon this clock, including the
> nanosleep() function; nor on the expiration of relative timers based
> upon this clock. Consequently, these time services shall expire when
> the requested relative interval elapses, independently of the new or
> old value of the clock.

sem_timedwait and pthread_mutex_timedlock specify timeouts in
absolute time, AFAICT.

This seems like such an obvious problem that I'm confused I haven't
seen any mention in the POSIX rationales.

What am I missing?

Regards.

[Casper H.S. Dik]

Noob <ro...@127.0.0.1> writes:

>sem_timedwait and pthread_mutex_timedlock specify timeouts in
>absolute time, AFAICT.

>This seems like such an obvious problem that I'm confused I haven't
>seen any mention in the POSIX rationales.

Perhaps this is one of the reasons why Solaris has:

int pthread_mutex_reltimedlock_np(pthread_mutex_t *restrict mutex,
const struct timespec *restrict rel_timeout);

(where the postfix "_np" says "non-posix")

However, it uses "CLOCK_REALTIME" was the clock for this function and
it would then, I expect, give the same problem.

Casper

[Rainer Weikusat]

Noob <ro...@127.0.0.1> writes:

[...]

>> CLOCK_MONOTONIC
>> The identifier for the system-wide monotonic clock, which is
>> defined as a clock measuring real time, whose value cannot be set
>> via clock_settime() and which cannot have negative clock jumps.
>> The maximum possible clock jump shall be implementation-defined.
>
> Question #2 : so how does one deal with the problem of using
> clock_settime, while several CLOCK_REALTIME-based time-outs
> are expected?

Conceptuelly, there no such problem: They're supposed to expired when
'the current time' has progressed to a specific datum. If some
'entity' changes the realtime clock via clock_settime to some value,
it is the problem of that someone to ensure that it is correct (no
matter how hard 'some people' try to ignore this, there is actually
something like 'the correct time' and it is independent of any
particular time measuring device).

BTW, assuming that

CLOCK_REALTIME
The identifier of the system-wide clock measuring real time.

and in absence of a device which cause a computer to travel back in
time, how can something which measures 'real time' have 'negative
clock jumps'? If it has, than, it obviously doesn't measure real time
since 'real time' never moves backwards. It is then rather a
particularly grotty PRNG.

[Barry Margolin]

In article <874nfeb...@sapphire.mobileactivedefense.com>,

Clocks can be wrong, and correcting it may require moving the hands back.

It sounds like your quibble is that this is called "real" time in the
macro. Nothing in software is really real, computers only know what
they're told, and humans can lie and make mistakes.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Noob]

Rainer Weikusat wrote:

> Noob wrote:
>
>>> CLOCK_MONOTONIC
>>> The identifier for the system-wide monotonic clock, which is
>>> defined as a clock measuring real time, whose value cannot be set
>>> via clock_settime() and which cannot have negative clock jumps.
>>> The maximum possible clock jump shall be implementation-defined.
>>
>> Question #2 : so how does one deal with the problem of using
>> clock_settime, while several CLOCK_REALTIME-based time-outs
>> are expected?
>

> Conceptually, there no such problem: They're supposed to expired when

> 'the current time' has progressed to a specific datum.

Thing is, I don't care about the "current time".

Conceptually, when I say "try to take this lock for 10 seconds,
either succeed or give up when the time is up", the current time
is not important, all I care about is that the operation should
not block more than 10 seconds (and not less than 10 seconds,
unless I have successfully taken the lock).

As a matter of fact, the POSIX language for clock_settime:

Setting the value of the CLOCK_REALTIME clock via clock_settime()
shall have no effect on threads that are blocked waiting for a
relative time service based upon this clock, including the
nanosleep() function; nor on the expiration of relative timers based
upon this clock. Consequently, these time services shall expire when
the requested relative interval elapses, independently of the new or
old value of the clock.

seems to allow platforms to provide an extension of sem_timedwait
and/or pthread_mutex_timedlock whereby the extended functions would
wait for a relative CLOCK_REALTIME-based duration.

I'm using Linux, are there any such extensions?

Otherwise, as a work-around, I suppose I could set a timer which
would fire a signal, and use a blocking operation, but I'd have
to deal with the races created by splitting the atomic op.

> If some
> 'entity' changes the realtime clock via clock_settime to some value,
> it is the problem of that someone to ensure that it is correct

Errr, so what do you propose? When the system boots, its notion
of the current time is completely wrong. We start the main app,
and "some time later" I manage to get a better idea of the
current time. I suppose I could leave the system in its wrong
view of space and time, but it interacts with the outside, and
I'm afraid that stuff like checking the validity of certificates
will mysteriously fail if the system is stuck in 1970, or whatever
the default is.

Regards.

[Rainer Weikusat]

Noob <root@localhost> writes:
> Rainer Weikusat wrote:
>
>> Noob wrote:
>>
>>>> CLOCK_MONOTONIC
>>>> The identifier for the system-wide monotonic clock, which is
>>>> defined as a clock measuring real time, whose value cannot be set
>>>> via clock_settime() and which cannot have negative clock jumps.
>>>> The maximum possible clock jump shall be implementation-defined.
>>>
>>> Question #2 : so how does one deal with the problem of using
>>> clock_settime, while several CLOCK_REALTIME-based time-outs
>>> are expected?
>>
>> Conceptually, there no such problem: They're supposed to expired when
>> 'the current time' has progressed to a specific datum.
>
> Thing is, I don't care about the "current time".
>
> Conceptually, when I say "try to take this lock for 10 seconds,
> either succeed or give up when the time is up", the current time
> is not important,

The current time is important because there's no way to determine if
'10 seconds' have passed without some concept of 'current time'. In
this particular case, the pthread_ routines wait based on 'absolute
time' because this means the wait can be interrupted and restarted
without invalidating the timeout argument as a side effect. The reason
why they refer to 'the realtime clock' is very likely because they were
conceived before different 'clock types' where invented and
consequently, the realtime clock was the only available option.

[Rainer Weikusat]

Barry Margolin <bar...@alum.mit.edu> writes:
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:

[...]

>> BTW, assuming that
>>
>> CLOCK_REALTIME
>> The identifier of the system-wide clock measuring real time.
>>
>> and in absence of a device which cause a computer to travel back in
>> time, how can something which measures 'real time' have 'negative
>> clock jumps'? If it has, than, it obviously doesn't measure real time
>> since 'real time' never moves backwards. It is then rather a
>> particularly grotty PRNG.
>
> Clocks can be wrong, and correcting it may require moving the hands
> back.

That would be the clock_settime part. But a device which, when left on
its own, produces a sequence of numbers where observing 'the current
number' at a time T enables no predictions about the number which
will be observable at some time T' > T is not a clock but a random
number generator and claiming that it 'measures real time' makes no
more sense that claiming that 'real time' can be measured by rolling a
dice.

[Jorgen Grahn]

On Wed, 2013-04-10, Noob wrote:
> Rainer Weikusat wrote:
...

>> If some
>> 'entity' changes the realtime clock via clock_settime to some value,
>> it is the problem of that someone to ensure that it is correct
>
> Errr, so what do you propose? When the system boots, its notion
> of the current time is completely wrong. We start the main app,
> and "some time later" I manage to get a better idea of the
> current time.

Sounds to me your startup order is incorrect, then? Just like some
daemons don't want to start until e.g. DNS or an MTA is available,
your software doesn't want to start until system time is roughly
correct (stops jumping).

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Eric Sosman]

On 4/10/2013 3:13 PM, Rainer Weikusat wrote:
> Noob <root@localhost> writes:
>> Rainer Weikusat wrote:
>>
>>> Noob wrote:
>>>
>>>>> CLOCK_MONOTONIC
>>>>> The identifier for the system-wide monotonic clock, which is
>>>>> defined as a clock measuring real time, whose value cannot be set
>>>>> via clock_settime() and which cannot have negative clock jumps.
>>>>> The maximum possible clock jump shall be implementation-defined.
>>>>
>>>> Question #2 : so how does one deal with the problem of using
>>>> clock_settime, while several CLOCK_REALTIME-based time-outs
>>>> are expected?
>>>
>>> Conceptually, there no such problem: They're supposed to expired when
>>> 'the current time' has progressed to a specific datum.
>>
>> Thing is, I don't care about the "current time".
>>
>> Conceptually, when I say "try to take this lock for 10 seconds,
>> either succeed or give up when the time is up", the current time
>> is not important,
>
> The current time is important because there's no way to determine if
> '10 seconds' have passed without some concept of 'current time'.

Have to disagree. The IBM S/360 on which I did much of my
early learning had an "interval timer," but no "clock." Lacking
a clock it had no notion of "current time,"[*] yet it could and
would measure a ten-second timeout by counting ticks. Timeouts
were not "Call me at half past," but "Call me in ten seconds."

[*] The various O/S'es simulated a clock service by asking
the operator to enter the date and time during boot-up, and by
counting ticks thereafter. But this dead reckoning was an output
of the interval measurement, not an input to it, and it didn't
underly the timeout services. (Also, you'd get some amusing
printouts when the operator mis-typed the time ...)

--
Eric Sosman
eso...@comcast-dot-net.invalid

[Rainer Weikusat]

Jorgen Grahn <grahn...@snipabacken.se> writes:
> On Wed, 2013-04-10, Noob wrote:
>> Rainer Weikusat wrote:
> ...
>>> If some
>>> 'entity' changes the realtime clock via clock_settime to some value,
>>> it is the problem of that someone to ensure that it is correct
>>
>> Errr, so what do you propose? When the system boots, its notion
>> of the current time is completely wrong. We start the main app,
>> and "some time later" I manage to get a better idea of the
>> current time.
>
> Sounds to me your startup order is incorrect, then? Just like some
> daemons don't want to start until e.g. DNS or an MTA is available,
> your software doesn't want to start until system time is roughly
> correct (stops jumping).

In case splitting 'the main app' into a 'pre-settime' and
'post-settime' part isn't easily possible, another idea would be to
record the fact that the clock was changed somewhere (eg, create a
file on 'some tmpfs) and then wrap the timedwait calls with some set
of functions which check if the clock was set and restart the call
with an adjusted timeout if it was started before the clock was set
and ETIMEDOUT was returned. At worst, this should result in the
timeout period being doubled, although this could probably be avoided
(at least if settime can be locked out while doing the calculation) by
recording the difference between the realtime clock and the monotonic
clock before the call and calculating a new timeout based on the
current difference between these two clocks, the old difference and
the current value of the realtime clock.

A 'global function pointer' per wrapped function could be used to
avoid this checks in subsequent calls: Always call a timedout routine
via this function pointer, initialize it to the wrapper routines, set
it to the POSIX functions after the clock was set.

[Philip Guenther]

On Wednesday, April 10, 2013 8:58:20 AM UTC-7, Casper H. S. Dik wrote:
> Noob <ro...@127.0.0.1> writes:
>
> >sem_timedwait and pthread_mutex_timedlock specify timeouts in
> >absolute time, AFAICT.
>
> >This seems like such an obvious problem that I'm confused I haven't
> >seen any mention in the POSIX rationales.

Try:
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xsh_chap02.html#tag_22_02_08_19

and scrolldown to "History of Resolution Issues". The text there would seem to imply that the predecessor or input to 1003.1d had some version of "mutex lock with *relative* timeout".

> Perhaps this is one of the reasons why Solaris has:
>
> int pthread_mutex_reltimedlock_np(pthread_mutex_t *restrict mutex,
> const struct timespec *restrict rel_timeout);
>
> (where the postfix "_np" says "non-posix")

(Hmm, isn't it normally expanded as "non-portable"? Then again, "non-posix" is probably a better description...)

> However, it uses "CLOCK_REALTIME" was the clock for this function and
> it would then, I expect, give the same problem.

Actually, no: POSIX specifies that only absolute timeouts against CLOCK_REALTIME are affected by clock_settime(). c.f the 5th and 6th paragraphs of the descriptions at
http://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_settime.html


Regarding the original poster's problem:

It seems that the standard's view of the use cases for pthread_mutex_timedlock(), sem_timedwait(), and mq_timed{send,receive}() didn't give much weight to the combination of:
* relative timeout desired
* clock adjustments via clock_settime() are large relative to the
timeout desired

At least that's how I interpret this sentence from the rationale I cited above:
When these functions [pthread_mutex_timedlock(), etc] are given
relative timeouts, the timeouts are typically for error recovery
purposes and need not be so precise.

Well, the large clock adjustment in this case means that "not so precise" == "really really *im*precise".


If I was in your position, couldn't arrange for the clock to get at least a gross adjustment before the application got to this point, and felt constrained to work within the standard, I would probably look first at changing the design such that threads block on a condition variable instead of a mutex in this case, and then (of course) use pthread_condattr_setclock() to set it to time against CLOCK_MONOTONIC. If that code-path is used throughout the lifetime of the process then a check of whether it introduces a excessive overhead or latency might be in order, depending on how exactly your design uses it.


Philip Guenther

[Johann Klammer]

Remove the things with timeout and replace it with calls to
mutex_lock/unlock, check the conditions yourself and use usleep() for
delaying/waiting. IMHO those *_timedwait thingys are broken...

[Barry Margolin]

In article <874nfei...@sapphire.mobileactivedefense.com>,

Rainer Weikusat <rwei...@mssgmbh.com> wrote:

> Noob <root@localhost> writes:
> > Rainer Weikusat wrote:
> >
> >> Noob wrote:
> >>
> >>>> CLOCK_MONOTONIC
> >>>> The identifier for the system-wide monotonic clock, which is
> >>>> defined as a clock measuring real time, whose value cannot be set
> >>>> via clock_settime() and which cannot have negative clock jumps.
> >>>> The maximum possible clock jump shall be implementation-defined.
> >>>
> >>> Question #2 : so how does one deal with the problem of using
> >>> clock_settime, while several CLOCK_REALTIME-based time-outs
> >>> are expected?
> >>
> >> Conceptually, there no such problem: They're supposed to expired when
> >> 'the current time' has progressed to a specific datum.
> >
> > Thing is, I don't care about the "current time".
> >
> > Conceptually, when I say "try to take this lock for 10 seconds,
> > either succeed or give up when the time is up", the current time
> > is not important,
>
> The current time is important because there's no way to determine if
> '10 seconds' have passed without some concept of 'current time'.

Of course there is. Imagine you have a stopwatch or a kitchen timer, but
no clock. You can't tell what time it is, but you can tell how much time
has passed since you started the timer.

[Barry Margolin]

In article <87vc7ug...@sapphire.mobileactivedefense.com>,

If it ticks at a known, consistent rate, you just have to count N ticks
to measure real time. Stable oscillators are old technology (but I
suggest avoiding a laptop with a pendulum).

[Rainer Weikusat]

Because the timer displays 'the current time' relative to the time
when it was started. That (and the current value of CLOCK_MONOTONIC)
is as 'current' as any other 'current time', it just isn't meaningful
except when referring to the particular clock the value came from
because it doesn't (except by accident) use the same frame of
reference as any other clock.

[Rainer Weikusat]

Barry Margolin <bar...@alum.mit.edu> writes:
> Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>
>> Barry Margolin <bar...@alum.mit.edu> writes:
>> > Rainer Weikusat <rwei...@mssgmbh.com> wrote:
>>
>> [...]
>>
>> >> BTW, assuming that
>> >>
>> >> CLOCK_REALTIME
>> >> The identifier of the system-wide clock measuring real time.
>> >>
>> >> and in absence of a device which cause a computer to travel back in
>> >> time, how can something which measures 'real time' have 'negative
>> >> clock jumps'? If it has, than, it obviously doesn't measure real time
>> >> since 'real time' never moves backwards. It is then rather a
>> >> particularly grotty PRNG.
>> >
>> > Clocks can be wrong, and correcting it may require moving the hands
>> > back.
>>
>> That would be the clock_settime part. But a device which, when left on
>> its own, produces a sequence of numbers where observing 'the current
>> number' at a time T enables no predictions about the number which
>> will be observable at some time T' > T is not a clock but a random
>> number generator and claiming that it 'measures real time' makes no
>> more sense that claiming that 'real time' can be measured by rolling a
>> dice.
>
> If it ticks at a known, consistent rate, you just have to count N ticks
> to measure real time.

Well, obviously. But this means the 'current tick count' must be
observable and it must actually be a tick counter, that is, something
whose value is monotonically increasing. Time can then be measured by
comparing the current value with some previously recorded value.

[Noob]

Johann Klammer wrote:

> Remove the things with timeout and replace it with calls to
> mutex_lock/unlock, check the conditions yourself and use usleep() for
> delaying/waiting. IMHO those *_timedwait thingys are broken...

(The calls to sem_timedwait and pthread_mutex_timedlock are not
in our code, but in library code.)

But even if I could change that code, I'm not sure I understand
what you're suggesting. Are you saying I should poll the lock,
and implement my own time out??

Are you suggesting that I replace
(pseudo-code, no error handling)

int err = pthread_mutex_timedlock(&mutex, 10 seconds);
if (err == ETIMEDOUT) { ... }

with

while ( 1 ) {
int err = pthread_mutex_trylock(&mutex);
if (err == 0) break;
if (we tried too many times, time out) break;
usleep(10 ms)
}

I hope not, because the latter code is awful.

Regards.

[Barry Margolin]

In article <87obdl1...@sapphire.mobileactivedefense.com>,

I.e. the CLOCK_MONOTONIC option that the OP wished for, which is
available in some other contexts.

[Rainer Weikusat]

The so-called 'monotonic clock' can't be used for measuring 'real
time' because it has no defined relation to it. Ie, one can't send a
message with a timestamp one got from the 'deluded, but boring' clock to
another system and compare the received timestamp with the current
value of the 'deluded but boring' clock on this system in order to
determine how long it took until the message arrived. This can only be
done by using a clock based on 'real time' (that would be UTC in this
case) and only if the "There is no network, no time, no world, no
other humans or Gods!!! Nothing but my all important, deluded mind
which travels backwards and forwards in time all the time !!!!!
Repent, all you dirty sinners with watches who mistakenly believe to
be using trains and bussess !!!!!! This thing you call world is about
to end !!!!" can, at some point in time, politely be made to
understand that nobody gives a damn for their deluded phantasies and
that a clock which runs backwards is broken (or a randon number
generator but not a clock).

[Rainer Weikusat]

Rainer Weikusat <rwei...@mssgmbh.com> writes:

[...]

> "There is no network, no time, no world, no other humans or Gods!!!
> Nothing but my all important, deluded mind which travels backwards
> and forwards in time all the time !!!!! Repent, all you dirty
> sinners with watches who mistakenly believe to be using trains and
> bussess !!!!!! This thing you call world is about
> to end !!!!"

Since it is pretty much guaranteed that one of the literarically
challenged "I'm as hard as the hard sciences" people will trip over
that: I won't claim that this is a particularly good poem but it is
nevertheless one. And it seemed an appropriate way to express the total
madness (IMHO) of the idea that, in an ever increasingly networked
world where lots of independent computers have to coordinate their
actions, the notion of 'a realtime clock' which actually behaves like
a clock (and can thus be used to 'measure real time') ought to be
abandoned.

Especially considering that the clock_*-routines are a fairly recent
invention and the only 'other way' which works on older systems is to
use gettimeofday instead.

[Johann Klammer]

I don't know about pthread_mutex_timedlock(there doesn't seem to be one
on this box), but I did replace a lot of my cond_timedwait()s with code
similar to the snippet you present above... I did do it because all the
gettimeofday() calls increased the load noticeably, not because of clock
setting... Still, it would avoid the timeout problem you describe...

[Rainer Weikusat]

Johann Klammer <klam...@NOSPAM.a1.net> writes:
>> Johann Klammer wrote:
>>> Remove the things with timeout and replace it with calls to
>>> mutex_lock/unlock, check the conditions yourself and use usleep() for
>>> delaying/waiting.

[...]

> I did replace a lot of my cond_timedwait()s with
> code similar to the snippet you present above... I did do it because
> all the gettimeofday() calls increased the load noticeably,

Could you perhaps provide some more detailed information about that,
eg what is this 'load'[*] gettimeofday calls affected and how you
determined that they affected it?

[*] The only definition of 'load' I'm aware of is "number processes in
runnable state which can't run ATM because resources are not
available". This doesn't seem to make sense in the context of the
gettimeofday statement.

[Johann Klammer]

top showed about 30% in the %CPU column with the gettimeofday, and
around 15% without... It was a bit of a specialized case... reading ts
packets from a dvb device and using a cond var to wait at the end of a
queue where they ended up. So gettimeofday got called _very_ often.
usleep variant slept and did processing in a slightly more bursty
fashion but reduced processor load...

[Scott Lurndal]

The "cost" of gettimeofday(2) differs by operating system. If the OS
has to cross the user/kernel boundary (e.g. a system call), the call will
be more expensive. Thus SVR4 implemented a mechanism that allowed the C
library to map, read-only, a page of the kernel address space that
contained the time-of-day value. This turned the library call and system call into a
library call and a memory reference. They did this specifically for
Oracle which used gettimeofday() extensively for timestamping.

Linux has adopted a similar approach, so gettimeofday() should be quite
efficient with modern c library/kernel combinations.

scott

[Johann Klammer]

Scott Lurndal wrote:
[...]

> Linux has adopted a similar approach, so gettimeofday() should be quite
> efficient with modern c library/kernel combinations.
>

Great, now I had to go look it up... can you tell me where in eglibc
that access would be? I came across a __gettimeofday in
posix/gettimeofday.c ... It uses time() for the seconds and sets the
usecs to zero(!)...

s390 uses a syscall
powerpc uses a syscall
x86_64 uses a syscall
aix uses inline assembler to read from some HW regs
alpha uses a syscall

similar for time()

[Jorgen Grahn]

On Thu, 2013-04-11, Scott Lurndal wrote:
> The "cost" of gettimeofday(2) differs by operating system. If the OS
> has to cross the user/kernel boundary (e.g. a system call), the call will
> be more expensive. Thus SVR4 implemented a mechanism that allowed the C

> library to map [...]

>
> Linux has adopted a similar approach, so gettimeofday() should be quite
> efficient with modern c library/kernel combinations.

As I understand it, it's somewhat architecture-dependant on Linux, and
also controlled by a sysctl (or a /proc entry? I forget, and my
Google skills failed). I use systems today where every gettimeofday()
ends up as a system call, visible in strace and all.

[Rainer Weikusat]

Jorgen Grahn <grahn...@snipabacken.se> writes:
> On Thu, 2013-04-11, Scott Lurndal wrote:
>> The "cost" of gettimeofday(2) differs by operating system. If the OS
>> has to cross the user/kernel boundary (e.g. a system call), the call will
>> be more expensive. Thus SVR4 implemented a mechanism that allowed the C
>> library to map [...]
>>
>> Linux has adopted a similar approach, so gettimeofday() should be quite
>> efficient with modern c library/kernel combinations.
>
> As I understand it, it's somewhat architecture-dependant on Linux, and
> also controlled by a sysctl (or a /proc entry? I forget, and my
> Google skills failed).

On Linux, 'a sysctl' is usually the same as 'modifying a
/proc/sys/...' file. For this case, I looked into /proc/sys/kernel for
likely suspects and found vsyscall64. Some documentation on that

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-General_System_Tuning-gettimeofday_speedup.html

(located via googling /proc/sys/kernel/vsyscall64)

[Nobody]

On Thu, 11 Apr 2013 20:19:49 +0200, Johann Klammer wrote:

> Scott Lurndal wrote:
> [...]
>> Linux has adopted a similar approach, so gettimeofday() should be quite
>> efficient with modern c library/kernel combinations.
>>
> Great, now I had to go look it up... can you tell me where in eglibc
> that access would be?

Here (Linux 3.6.11, x86-64), the functions time(), gettimeofday(),
clock_gettime() and getcpu() are implemented in linux-vdso, a virtual
shared library which is embedded in the kernel and automatically mapped
into each process' address space.

[Casper H.S. Dik]

sc...@slp53.sl.home (Scott Lurndal) writes:

>The "cost" of gettimeofday(2) differs by operating system. If the OS
>has to cross the user/kernel boundary (e.g. a system call), the call will
>be more expensive. Thus SVR4 implemented a mechanism that allowed the C
>library to map, read-only, a page of the kernel address space that
>contained the time-of-day value. This turned the library call and system call into a
>library call and a memory reference. They did this specifically for
>Oracle which used gettimeofday() extensively for timestamping.

You will still need to read both of the numbers safely (i.e., getting the
second and the micro second from the same tuple).

I wasn't aware of that implementation, but Solaris has been using
a "fasttrap" (still crosses into the kernel but without the overhead
generally associated with system call overhead) Even though SVR4 was
one of Solaris ancestors, I have never even seen traces of the "shared
memory" mapped in all processes.

>Linux has adopted a similar approach, so gettimeofday() should be quite
>efficient with modern c library/kernel combinations.

Although I really wonder what algorithms want to know what time it is *now*
rather than the lasttime gettimeofday() was called.

Casper

[Casper H.S. Dik]

Jorgen Grahn <grahn...@snipabacken.se> writes:

>As I understand it, it's somewhat architecture-dependant on Linux, and
>also controlled by a sysctl (or a /proc entry? I forget, and my
>Google skills failed). I use systems today where every gettimeofday()
>ends up as a system call, visible in strace and all.

The Solaris fasttrap is indeed not visible with truss (similar to
strace), so gettimeofday(), time() [ in newer versions of Solaris ],
and gethrtime() are not visible.

Casper

[Jorgen Grahn]

OK.

Clarifying my statement above, just in case: I use /Linux/ systems
which have gettimeofday() as a system call. What I meant to convey
was: if you use a modern, well-tuned Linux distro on x86_64 hardware
the optimization is most likely in effect, but anything more exotic
you'd better check. (The system I mentioned is 64-bit kernel and
32-bit userspace.)

[Scott Lurndal]

Casper H.S. Dik <Caspe...@OrSPaMcle.COM> writes:
>sc...@slp53.sl.home (Scott Lurndal) writes:
>
>>The "cost" of gettimeofday(2) differs by operating system. If the OS
>>has to cross the user/kernel boundary (e.g. a system call), the call will
>>be more expensive. Thus SVR4 implemented a mechanism that allowed the C
>>library to map, read-only, a page of the kernel address space that
>>contained the time-of-day value. This turned the library call and system call into a
>>library call and a memory reference. They did this specifically for
>>Oracle which used gettimeofday() extensively for timestamping.
>
>You will still need to read both of the numbers safely (i.e., getting the
>second and the micro second from the same tuple).
>
>I wasn't aware of that implementation, but Solaris has been using
>a "fasttrap" (still crosses into the kernel but without the overhead
>generally associated with system call overhead) Even though SVR4 was
>one of Solaris ancestors, I have never even seen traces of the "shared
>memory" mapped in all processes.

IIRC, it appeared in SVR4.2MP which was post the sun/usl work on SVR4.

scott

[James K. Lowden]

On Wed, 10 Apr 2013 17:08:24 +0200
Noob <ro...@127.0.0.1> wrote:

> I write software for set-top boxes. When they are powered-on,
> they have no idea what time it is. They acquire the current
> time and date over a satellite link. By the time we're ready
> to read from the satellite tuner, we're pretty far along in
> the board's init.
>
> Several software components use semaphores and mutexes, some with
> a time-out value. At some point, we acquire the correct time and
> date, and we set the system time -- BOOM. Every time-out fires at
> the same time.

Does this happen if you use poll(2)? AFAICT nothing in the definition
of poll depends on the time of day. If poll times out when the sysadmin
changes the date, I would say that's a bug in the implementation or the
documentation.

--jkl