sem_timedwait use with the MONOTONIC clock?

[bob.fe...@rafresearch.com]

I am using the Nuttx monotonic clock in my application.

I am setting up the timespec using CLOCK_MONOTONIC and
CONFIG_CLOCK_MONOTONIC=y
is in the .config file.

sem_timedwait() is returning immediately.

I looked in sched/semaphore/sem_timedwait.c and found...

 status = clock_abstime2ticks(CLOCK_REALTIME, abstime, &ticks);

Shouldn't there be an #ifdef that selects CLOCK_MONOTONIC when it's configured?

Has clock_abstime2ticks() been tested using CLOCK_MONOTONIC?

-Bob

[Gregory Nutt]

The concept of an absolute time with CLOCK_MONOTONIC is a little nonsensical.  CLOCK_REALTIME and CLOCK_MONOTONIC are based on the same system timer that starts at zero and increments at a rate determined by CONFIG_USEC_PER_TICK.  CLOCK_REALTIME has a offset that adjusts this counter to produce a real wall clock time.  So you can talk about an absolute date/time with CLOCK_REALTIME, but not with CLOCK_MONOTONIC.  The concept is meaningless.

There are no guarantees about about the meaning of CLOCK_MONOTONIC other than, as its name suggests, in increments monotonically:  It nevers goes backward in time.

Both clocks can be used used for time differences, but only CLOCK_REALTIME can be used for absolute time.  CLOCK_MONTONIC is preferable for time differences between times sampled at different events.  That is because someone might change CLOCK_REALTIME (for example, with the NSH date command).  In that case, CLOCK_REALTIME may, indeed, go backward.

So, no clock_abstime2ticks() has never been tested with CLOCK_MONOTONIC and nor do I suspect that it would work.

[patacongo]

So, no clock_abstime2ticks() has never been tested with CLOCK_MONOTONIC and nor do I suspect that it would work.

Looking at the code, I don't see any reason why clock_abstime2tick would not work if provided with a valid  absolute time obtained from the same clock (which is, of course, a meaningless time)

I suspect that when you are calling sem_timedwait(), you are are not using a valid absolute time from CLOCK_REALTIME.  If you use a nonsense "absolute"  time from CLOCK_MONOTONIC, with sem_timedwait(), that will absolutely fail.

POSIX requires that the absolute time provided to sem_timedwait() derive from CLOCK_REALTIME.  See http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_timedwait.html  No other clock source is valid for use with sem_timedwait().

[bob.fe...@rafresearch.com]

It seems to me that running sem_timedwait() using CLOCK_REALTIME would be very problematic.
Embedded systems often have to slave their "system time" to an external entity, such as network time or in our case GPS time.
Our chips oscillator will drift with respect to the external time base, so we resync our "system time" periodically. The tail of these corrections goes over a hundreds of milliseconds. If the local "system time" falls behind the external reference, say by about 150 ms, then when the time is resynchronized CLOCK_REALTIME will jump ahead by those 150 ms and any outstanding sem_timedwait() actions would be impacted.

We use CLOCK_MONOTONIC for time that must be monotonic and CLOCK_REALTIME for time that must be in sync with our external time base.
Are we using the wrong mechanism for time synchronization?

-Bob

[bob.fe...@rafresearch.com]

Typo error...

corrections goes over a hundreds of milliseconds

should be...
corrections goes over a hundred of milliseconds

-Bob

[Gregory Nutt]

It seems to me that running sem_timedwait() using CLOCK_REALTIME would be very problematic.
Embedded systems often have to slave their "system time" to an external entity, such as network time or in our case GPS time.
Our chips oscillator will drift with respect to the external time base, so we resync our "system time" periodically. The tail of these corrections goes over a hundreds of milliseconds. If the local "system time" falls behind the external reference, say by about 150 ms, then when the time is resynchronized CLOCK_REALTIME will jump ahead by those 150 ms and any outstanding sem_timedwait() actions would be impacted.

We use CLOCK_MONOTONIC for time that must be monotonic and CLOCK_REALTIME for time that must be in sync with our external time base.
Are we using the wrong mechanism for time synchronization?

No, but per the POSIX standard, the *absolute time* provided to sem_timedwait() must be based on CLOCK_REALTIME.  That is a POSIX requirement and you must conform to it if you wish to use a POSIX operating system.

The time that you provide to sem_timedwait() is a absolute time meaning an exact time in the future:  For example, next Tuesday at 3:07.123 pm UTC.  The time is, internally, based on an a relative time calculated when sem_timewait() is called and the relative time when the timer fires will not change if you modify the system time (CLOCK_REALTIME).  But that is an implementation detail and may not be how it works on other systems.

[patacongo]

No, but per the POSIX standard, the *absolute time* provided to sem_timedwait() must be based on CLOCK_REALTIME.  That is a POSIX requirement and you must conform to it if you wish to use a POSIX operating system.

If you really have an issue with the POSIX definition, you should take this up with OpenGroup.org.  My goal is to implement the POSIX standard and accurately and as thoroughly as possible on these embedded platforms.  There are forums there for discussion of the standards.  If you can convince, OpenGroup, IBM, Linux, Apple, QNX, Green Hills, etc. that they should change the POSIX standard I will, of course, change NuttX to match the revised POSIX standard.

Otherwise, there is not a snowball's chance in hell that I will modify the NuttX behavior.

[Alan Carvalho de Assis]

Hi Bob,

I will need to use time resyncronization on my next project and I want
to understand better the issue you are facing.

Is your problem similar to it:
https://stackoverflow.com/questions/29153310/how-would-i-cope-with-changes-to-the-system-time-while-doing-sem-timedwait

Could you please supply more detail?

BR,

Alan

> --
> You received this message because you are subscribed to the Google Groups
> "NuttX" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nuttx+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Gregory Nutt]

I will need to use time resyncronization on my next project and I want
to understand better the issue you are facing.

Is your problem similar to it:
https://stackoverflow.com/questions/29153310/how-would-i-cope-with-changes-to-the-system-time-while-doing-sem-timedwait

As I mentioned in a previous email, the NuttX sem_timedwait() converts the absolute time to a relative time in system clock ticks.  The timing is then controlled by a NuttX wdog timer which does not depend on the either CLOCK_s.

Normally this is wasteful because you really want a relative time delay anyway.  So the application has to create the absolute time be getting the currently time (using CLOCK_REALTIME), then adding a delay to that to get the corresponding absolute time.  NOTE that there is a race condition in this.  You could be suspended after getting the current time, then apply the delay and create a relative delay that is less than what you wanted.

The use of relative times and wdog times within sem_timedwait() means that if you change the system while doing sem_timedwait(), nothing will change.  The timer will still go off at the same relative time delay.

But if you do get suspended between getting the current time, adding the delay and calling sem_timedwait(), then there is a possibility that the CLOCK_REALTIME could change before sem_timedwait() is called.  You could avoid that be disabling pre-emption:

1. call sched_lock()
2. call clock_gettime()
3. add realative time
4. call sem_timedwait()
5. call sched_unlock()

Greg

[bob.fe...@rafresearch.com]

Your statement...

The time is, internally, based on an a relative time calculated when sem_timewait() is called and the relative time when the timer fires will not change if you modify the system time (CLOCK_REALTIME).

eliminates my concern regarding the use of CLOCK_REALTIME in our timed semaphores.

I would be interested in finding out why POSIX mandates that sem_timedwait() must be based on CLOCK_REALTIME. I can think of reasons why timed semaphores should be implemented with a relative delay (such as in our application) and with an absolute UTC end time. Given the POSIX specification, it seems that an OS can implement sem_timedwait() to satisfy either of these cases, but not both.

It would seem that 99.9% of sem_timedwait() use desires a relative watchdog timer of a few seconds or less.
It's odd that the POSIX specification is worded to seem to favor the 0.1% case.
Thanks for supporting the 99.9% usage.

-Bob


On Thursday, October 11, 2018 at 11:42:50 PM UTC-7, bob.fe...@rafresearch.com wrote:

[bob.fe...@rafresearch.com]

Hi Alan,
We have several tasks that use queues to make work requests of other tasks. To get a worker task's attention, we place the work request in its queue and perform a sem_timedwait() with a short timeout (100 ms). Given the work handling capability of the worker task, that should be far more than sufficient for a response. And its a short enough time to take some recovery action before something bad happens, if the worker task is hung.

We are using a MCU chip with a tunable internal RC oscillator (we believe that the clock error over short periods of time should be less than 0.1%) This is more than accurate enough for the work we are performing except for one scenario. If we lose GPS contact, given this oscillator, our system time could drift 60 ms per minute. We know that this time drift is acceptable for the accuracy of the work we are doing, but it is unacceptable for a GPS outage of 2 minutes to cause sem_timedwait() to declare a false timeout when we reconnect GPS. (There could be a dozen sem_timedwait() all falsely declaring "timeout" at the same time.)

We are not using preemption in our application, so Greg's relative tick implementation of sem_timedwait() should work out fine.

-Bob

[Alan Carvalho de Assis]

Thank you Greg for this suggestion I will try to implement following your approach.

BR,

Alan

--
You received this message because you are subscribed to the Google Groups "NuttX" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+unsubscribe@googlegroups.com.

[Alan Carvalho de Assis]

Hi Bob,

Thank you for detailed explanation about your issue.

My needs are a little bit different: my devices will be sleeping  most part of the time and then it wake-up it will communicate with the server to transmit data and to sync the clock.

Also because I will use an external RTC with -/+2ppm its drift will be smaller.

BR,

Alan

To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+unsubscribe@googlegroups.com.

[Gregory Nutt]

> My needs are a little bit different: my devices will be sleeping  most
> part of the time and then it wake-up it will communicate with the
> server to transmit data and to sync the clock.
>
> Also because I will use an external RTC with -/+2ppm its drift will be
> smaller.

Just a note about use of the RTC.  The RTC is read by the system
automatically *only* at power up.  It is used only to set the time.
After that the time is driven by the system timer, possible SysTick
interrupts?

After waking up from deep sleep states, you will need to resynchronize
the system timer with the RTC time with your logic by calling the
function clock_synchronize().

You will need a good, stable crystal driven clock source to keep the
time stable using the system timer.  If it is not stable enough, you
make need to call clock_synchronize() periodically to keep the times in
synch.

Greg