uv_timer_start() don't use timerfd_settime() on Linux.

[Jean-Christian de Rivaz]

Hello,

I tried to use libuv on Linux for an application that require a precise periodicity. I observed that uv_timer_start(, ,100, 100) make the repetitive epoll() always wait a bit more than 100ms, regardless of the callback function execution time (strace -T):

epoll_wait(5, {}, 1024, 99)             = 0 <0.099287>  <= First epoll() wait 99.287ms
sync()                                  = 0 <0.054469>
epoll_wait(5, {}, 1024, 100)            = 0 <0.100211>  <= Second epoll() wait 100.211ms
sync()                                  = 0 <0.040866>
epoll_wait(5, {}, 1024, 100)            = 0 <0.100207>  <===
sync()                                  = 0 <0.046870>
epoll_wait(5, {}, 1024, 100)            = 0 <0.100214>  <===
sync()                                  = 0 <0.043035>
epoll_wait(5, {}, 1024, 100)            = 0 <0.100139>  <===

I use the sync() in the callback to very simply add a execution of about from 40ms to 50ms on the test machine, the final application is more complex than that, this is just to keep the example as simple as possible. After the first epoll(), all epoll() have a similar length of a bit more that 100ms. This already too long delay is in addition to the delay of the sync() call. In fine, the period of this example is between 140ms and 150ms instead of the expected 100ms. If I filter the sync() call time I get this (strace -tt):

00:31:27.960802 sync()                  = 0
00:31:28.106703 sync()                  = 0
00:31:28.250530 sync()                  = 0
00:31:28.391060 sync()                  = 0
00:31:28.529861 sync()                  = 0
00:31:28.669751 sync()                  = 0
00:31:28.810477 sync()                  = 0
00:31:28.949302 sync()                  = 0

The jitter of the sync() call time is very big, probably equal to the callback execution time.

I tested timerfd_create() on the same machine with a loop of read() on the timerfd file and sync(). The result is very different (strace -T):

timerfd_create(CLOCK_MONOTONIC, 0)      = 9 <0.000013>
timerfd_settime(9, 0, {it_interval={0, 100000000}, it_value={0, 100000000}}, NULL) = 0 <0.000014>
read(9, "\1\0\0\0\0\0\0\0", 8)          = 8 <0.100138>  <== First read() on timerd fd file wait 100.138ms
sync()                                  = 0 <0.053847>
read(9, "\1\0\0\0\0\0\0\0", 8)          = 8 <0.045944>  <= Second read() wait 45.944ms
sync()                                  = 0 <0.047778>
read(9, "\1\0\0\0\0\0\0\0", 8)          = 8 <0.051987>  <===
sync()                                  = 0 <0.046731>
read(9, "\1\0\0\0\0\0\0\0", 8)          = 8 <0.053102>  <===
sync()                                  = 0 <0.044829>
read(9, "\1\0\0\0\0\0\0\0", 8)          = 8 <0.055018>  <===

The read() delay compensate exactly the sync() delay to make a periodicity of exactly 100ms. If I filter the sync() call time I get this (strace -tt):

00:47:23.687205 sync()                  = 0
00:47:23.787220 sync()                  = 0
00:47:23.887393 sync()                  = 0
00:47:23.987276 sync()                  = 0
00:47:24.087210 sync()                  = 0
00:47:24.187164 sync()                  = 0
00:47:24.287242 sync()                  = 0
00:47:24.387330 sync()                  = 0
00:47:24.487286 sync()                  = 0
00:47:24.587181 sync()                  = 0
00:47:24.687237 sync()                  = 0

The jitter of the sync() call time is smaller than 1ms, something that the current libuv is unable to do.

I have two questions:

1) is there anything that would prevent libuv from using timerfd_settime() on Linux ?

2) Is there any motivation to do that ?

Best Regards,

Jean-Christian

[Saúl Ibarra Corretgé]

Hi Jean-Christian,

On 09/29/2014 01:06 AM, Jean-Christian de Rivaz wrote:
> Hello,
>
> I tried to use libuv on Linux for an application that require a precise
> periodicity. I observed that uv_timer_start(, ,100, 100) make the
> repetitive epoll() always wait a bit more than 100ms, regardless of the
> callback function execution time (strace -T):
> |
> epoll_wait(5,{},1024,99) =0<0.099287> <=Firstepoll()wait 99.287ms
> sync() =0<0.054469>
> epoll_wait(5,{},1024,100) =0<0.100211> <=Secondepoll()wait
> 100.211ms
> sync() =0<0.040866>
> epoll_wait(5,{},1024,100) =0<0.100207> <===
> sync() =0<0.046870>
> epoll_wait(5,{},1024,100) =0<0.100214> <===
> sync() =0<0.043035>

> epoll_wait(5,{},1024,100) =0<0.100139> <===

> |
> I use the sync() in the callback to very simply add a execution of about
> from 40ms to 50ms on the test machine, the final application is more
> complex than that, this is just to keep the example as simple as
> possible. After the first epoll(), all epoll() have a similar length of
> a bit more that 100ms. This already too long delay is in addition to the
> delay of the sync() call. In fine, the period of this example is between
> 140ms and 150ms instead of the expected 100ms. If I filter the sync()
> call time I get this (strace -tt):
> |
> 00:31:27.960802sync() =0
> 00:31:28.106703sync() =0
> 00:31:28.250530sync() =0
> 00:31:28.391060sync() =0
> 00:31:28.529861sync() =0
> 00:31:28.669751sync() =0
> 00:31:28.810477sync() =0
> 00:31:28.949302sync() =0
> |
> The jitter of the sync() call time is very big, probably equal to the
> callback execution time.
>
> I tested timerfd_create() on the same machine with a loop of read() on
> the timerfd file and sync(). The result is very different (strace -T):
> |
> timerfd_create(CLOCK_MONOTONIC,0) =9<0.000013>

> timerfd_settime(9,0,{it_interval={0,100000000},it_value={0,100000000}},NULL)=0<0.000014>

> read(9,"\1\0\0\0\0\0\0\0",8) =8<0.100138> <==Firstread()on
> timerd fd file wait 100.138ms

> sync() =0<0.053847>

> read(9,"\1\0\0\0\0\0\0\0",8) =8<0.045944> <=Secondread()wait
> 45.944ms

> sync() =0<0.047778>

> read(9,"\1\0\0\0\0\0\0\0",8) =8<0.051987> <===

> sync() =0<0.046731>

> read(9,"\1\0\0\0\0\0\0\0",8) =8<0.053102> <===

> sync() =0<0.044829>
> read(9,"\1\0\0\0\0\0\0\0",8) =8<0.055018> <===

> |
> The read() delay compensate exactly the sync() delay to make a
> periodicity of exactly 100ms. If I filter the sync() call time I get
> this (strace -tt):
> |
> 00:47:23.687205sync() =0
> 00:47:23.787220sync() =0
> 00:47:23.887393sync() =0
> 00:47:23.987276sync() =0
> 00:47:24.087210sync() =0
> 00:47:24.187164sync() =0
> 00:47:24.287242sync() =0
> 00:47:24.387330sync() =0
> 00:47:24.487286sync() =0
> 00:47:24.587181sync() =0
> 00:47:24.687237sync() =0
> |
> The jitter of the sync() call time is smaller than 1ms, something that
> the current libuv is unable to do.
>

Periodic timers are a bit hard. Some people expect that the timer
compensates for the time spent in the callback, some don't cause they
want the interval to be preserved across callbacks. It's also quite
messy to handle the case in which the callback takes longer than the
next interval.

Having that said, timers on libuv don't compensate for the amount of
time spent in the callback.

> I have two questions:
> 1) is there anything that would prevent libuv from
> using timerfd_settime() on Linux ?

There are a couple I can think of: timerfd was added in Linux 2.6.22,
and I believe we still support older kernels, but more importantly,
timerfd would create a file descriptor per timer, that's one million fds
if we have one million timers, vs 0 fds which the current implementation
uses.

> 2) Is there any motivation to do that ?
>

Hopefully others chime in as well, but I'm not really inclined to go
this route, for the aforementioned reasons. I'm open to be proven
otherwise though :-)


Regards,

--
Saúl Ibarra Corretgé
bettercallsaghul.com

[Saúl Ibarra Corretgé]

Also, on top the above, creating and processing a timer requires 3
syscalls: timerfd_create, timerfd_gettime and read vs 0 if we keep the
min-heap.

[Iñaki Baz Castillo]

2014-09-29 11:00 GMT+02:00 Saúl Ibarra Corretgé <sag...@gmail.com>:
> Also, on top the above, creating and processing a timer requires 3
> syscalls: timerfd_create, timerfd_gettime and read vs 0 if we keep the
> min-heap.

Given that UV periodic timer is just an API sugar over the regular
timer, would not make sense that the loop execution time is taken into
account when setting the next timer period?

This is:

- Run periodic timer with interval 100 ms.
- First loop iteration takes 15 ms.
- Somehow libuv knows that and set the timer next period on 85 ms.

Just wondering.


--
Iñaki Baz Castillo
<i...@aliax.net>

[Jean-Christian de Rivaz]

Hi Saúl,

Also, on top the above, creating and processing a timer requires 3
syscalls: timerfd_create, timerfd_gettime and read vs 0 if we keep the
min-heap.

 

Delegation of the timers to the kernel will certainly free libuv from some code in addition to bring a far better precision. The timerfd_* syscalls was not added without reason into the Linux kernel.
The read() is not required if you don't want the number of expirations that have occurred. The integration of the file descriptor into the existing epoll() syscall is enough.

Regards,

Jean-Christian

[Jean-Christian de Rivaz]

Hi Iñaki,

Le lundi 29 septembre 2014 11:10:27 UTC+2, Iñaki Baz Castillo a écrit :

Given that UV periodic timer is just an API sugar over the regular
timer, would not make sense that the loop execution time is taken into
account when setting the next timer period?

This is:

- Run periodic timer with interval 100 ms.
- First loop iteration takes 15 ms.
- Somehow libuv knows that and set the timer next period on 85 ms.

Just wondering.

This is how periodic timers was handled in many libraries and applications before the timerfd_* syscall introduction, but proved to be very difficult to get a small jitter and to compensate the phase over time.

Best Regards,

Jean-Christian

[Saúl Ibarra Corretgé]

Hi!

It won't free libuv of any code, it would actually make it more complex,
because timerfd is linux only, so we need to keep the current code (or
find some timerfd equivalent) on all other Unix platforms.


Regards,

[Iñaki Baz Castillo]

Clear, thanks a lot for clarifying it.

[Ben Noordhuis]

On Mon, Sep 29, 2014 at 8:24 AM, Saúl Ibarra Corretgé <sag...@gmail.com> wrote:
> timerfd would create a file descriptor per timer, that's one million fds
> if we have one million timers, vs 0 fds which the current implementation
> uses.

That's not strictly necessary. Libuv could use a per-event loop
timerfd and multiplex timers on top of that.

There would be some system call overhead - an extra timerfd_settime()
call before some/most calls to epoll_wait() - but it's probably not
hugely expensive.

[Saúl Ibarra Corretgé]

I understand we could achieve sub-millisecond precision like that,
right? But it wouldn't "fix" the initial exposed problem here since we
still don't account for the duration of the callback... unless I missed
something :-)

[Iñaki Baz Castillo]

2014-09-29 14:11 GMT+02:00 Saúl Ibarra Corretgé <sag...@gmail.com>:
> I understand we could achieve sub-millisecond precision like that,
> right? But it wouldn't "fix" the initial exposed problem here since we
> still don't account for the duration of the callback... unless I missed
> something :-)

AFAIK epoll/kqueue/etc is needed to be notified about timerfd events,
right? If so we have the same issue since it may take some time since
timerfd fires until libuv checks it (exactly the current callback(s)
execution time). How to deal with it?

[Jean-Christian de Rivaz]

Hi Iñaki,

If the application is quick enough to call epoll before a timerfd fire, the timerfd grant a high precision and phase for periodic fire.

If the application is slow timerfd grand that futures periods are still in phase with the initial call.

If the application is so slow that it completely miss a period, the number of expirations are in the uint64_t returned by the read() on the timerfd file.

[Ben Noordhuis]

I have no opinion on whether you should account for callback time, I
just wanted to point out that it's possible to use timerfds without
undue overhead.

A clear reason for using a timerfd is that timerfds are usually more
precise than epoll_wait() with its default 50 us timeout slack.

(You can adjust the slack with the PR_SET_TIMERSLACK prctl() but you
can never reset it to exactly zero because that resets it to the timer
slack inherited from the parent process or thread.)

Apart from slack, the timeout argument to epoll_wait() is a lower
bound; the system call may sleep for longer than that. Timerfds give
you better (albeit not perfect) control over the sleep period.

[Sam Roberts]

I wonder if there is a conflict in use case between general purpose
timers, of which there can be many, and high precision timers, of
which maybe you only want a few.

The OP can use create timer fds himself, and add them to the uv loop
to be watched, to get the precision he wants within uv, I think.

Perhaps if a perfect API can't be found in uv, a new higher precision
timerfd based API with caveats about how many of these you can/should
create would be useful?

[Jean-Christian de Rivaz]

Hi Sam,

Your idea is interesting. After reading the reactions on my initial post, I wonder if it's a constructive proposal to split the question into multiples aspects:

1) Should libuv offer API for periodic timer ? The actual implementation do not and in fact correspond to a one shot timer restarted at the end of the callback. If the response is no, then actual API and documentation should be changed. If the response is yes, the implementation should be changed. I hope that the libuv community found an advantage to respond yes to this question, as periodic timers is certainly a common feature request and libuv is in best place to implement it.

2) Should libuv use timerfd to implement the aspect 1 in case the response is yes ? Certainly no in general as timerfd is usable only on Linux starting from kernel 2.6.23 or so. Depending on the performances of the general code for periodic timer (if there is any) timerfd might allow an improvement, but this is too early to be certain about that at this stage.

3) Should libuv offer a high precision timer API ? From a API point of view, it would be better to have a single API that scale and stay precise. An example on how to use timerfd watcher will be an immediate advantage for not too old Linux kernel users.

Best Regards.

Jean-Christian

[Saúl Ibarra Corretgé]

On 09/30/2014 08:13 AM, Jean-Christian de Rivaz wrote:
> Le mardi 30 septembre 2014 07:18:07 UTC+2, Sam Roberts a écrit :
>
> I wonder if there is a conflict in use case between general purpose
> timers, of which there can be many, and high precision timers, of
> which maybe you only want a few.
>
> The OP can use create timer fds himself, and add them to the uv loop
> to be watched, to get the precision he wants within uv, I think.
>
> Perhaps if a perfect API can't be found in uv, a new higher precision
> timerfd based API with caveats about how many of these you can/should
> create would be useful?
>
>
> Hi Sam,
>
> Your idea is interesting. After reading the reactions on my initial
> post, I wonder if it's a constructive proposal to split the question
> into multiples aspects:
>

Yeah, this is something we can definitely discuss. Let me get back to
you on this. Others feel free to comment as well!


Regards,

[Bert Belder]

Your idea is interesting. After reading the reactions on my initial post, I wonder if it's a constructive proposal to split the question into multiples aspects:

1) Should libuv offer API for periodic timer ? The actual implementation do not and in fact correspond to a one shot timer restarted at the end of the callback. If the response is no, then actual API and documentation should be changed. If the response is yes, the implementation should be changed. I hope that the libuv community found an advantage to respond yes to this question, as periodic timers is certainly a common feature request and libuv is in best place to implement it.

I don't think so. The current model was inherited from libev, which had no concept of "requests" like libuv does, instead it had only "handles".
Also the earlier versions of libuv didn't support `uv_cancel(uv_req_t*)` so at the time we had to make a timer a handle to support stopping the timer.

Saùl, Ben and myself already had some sort of informal agreement that a next iteration of libuv would only support one-shot timers with an API similar to this:
`uv_timeout(uv_timeout_req_t* timeout, int64_t timeout, uv_timeout_cb callback);

2) Should libuv use timerfd to implement the aspect 1 in case the response is yes ? Certainly no in general as timerfd is usable only on Linux starting from kernel 2.6.23 or so. Depending on the performances of the general code for periodic timer (if there is any) timerfd might allow an improvement, but this is too early to be certain about that at this stage.

I don't think we should use timerfd for uv_timer / uv_timeout timers. These timers are guaranteed to be fast and highly scalable, not precise. If we use timerfd on some linux flavors, they will be slow-and-precise on some operating systems, and fast-and-coarse on others. It makes libuv as a whole less useful.
 

3) Should libuv offer a high precision timer API ? From a API point of view, it would be better to have a single API that scale and stay precise. An example on how to use timerfd watcher will be an immediate advantage for not too old Linux kernel users.

Maybe - I like to think use cases; what are you trying to do? Writing a game loop, rendering video, synchonizing clocks?

- Bert

[Jean-Christian de Rivaz]

Le mardi 30 septembre 2014 13:52:29 UTC+2, Bert Belder a écrit :

Your idea is interesting. After reading the reactions on my initial post, I wonder if it's a constructive proposal to split the question into multiples aspects:

1) Should libuv offer API for periodic timer ? The actual implementation do not and in fact correspond to a one shot timer restarted at the end of the callback. If the response is no, then actual API and documentation should be changed. If the response is yes, the implementation should be changed. I hope that the libuv community found an advantage to respond yes to this question, as periodic timers is certainly a common feature request and libuv is in best place to implement it.

I don't think so. The current model was inherited from libev, which had no concept of "requests" like libuv does, instead it had only "handles".
Also the earlier versions of libuv didn't support `uv_cancel(uv_req_t*)` so at the time we had to make a timer a handle to support stopping the timer.

Saùl, Ben and myself already had some sort of informal agreement that a next iteration of libuv would only support one-shot timers with an API similar to this:
`uv_timeout(uv_timeout_req_t* timeout, int64_t timeout, uv_timeout_cb callback);

Ok, this has a least the advantage to make the situation more clear and don't prevent the potential addition of periodic timers in the future.

 

2) Should libuv use timerfd to implement the aspect 1 in case the response is yes ? Certainly no in general as timerfd is usable only on Linux starting from kernel 2.6.23 or so. Depending on the performances of the general code for periodic timer (if there is any) timerfd might allow an improvement, but this is too early to be certain about that at this stage.

I don't think we should use timerfd for uv_timer / uv_timeout timers. These timers are guaranteed to be fast and highly scalable, not precise. If we use timerfd on some linux flavors, they will be slow-and-precise on some operating systems, and fast-and-coarse on others. It makes libuv as a whole less useful.

Agree.

 

3) Should libuv offer a high precision timer API ? From a API point of view, it would be better to have a single API that scale and stay precise. An example on how to use timerfd watcher will be an immediate advantage for not too old Linux kernel users.

Maybe - I like to think use cases; what are you trying to do? Writing a game loop, rendering video, synchonizing clocks?

My actual application need to extract data from a hardware at a maximum interval of less than 100ms, but ideally about 50ms. The hardware already no a nice work in buffering the data in a large DMA but the tricky part is that it need a precise synchronization with the TAI maintained by the Linux kernel using a GPS. For this I need the low jitter and correct phase provided by timerfd. I don't strictly need libuv for this application, but this would help as on the other end there will be file and TCP streams feed by the data. Node.je is also running on the system for the configuration, making libuv a common library.

- Jean-Christian

[Iñaki Baz Castillo]

2014-09-30 13:52 GMT+02:00 Bert Belder <bertb...@gmail.com>:
> next iteration of libuv would only support one-shot timers with an API
> similar to this:
> `uv_timeout(uv_timeout_req_t* timeout, int64_t timeout, uv_timeout_cb
> callback);

While that is great it also completely drops the "periodic timer"
feature from libuv. Ok, current periodic timers are wrong but, should
libuv provide correct periodic timers (those in which the callback
execution time does not affect the next execution)?

By just providing the uv_timeout() request, users cannot implement
real periodic timers.

[Bert Belder]

On Tuesday, September 30, 2014 2:25:37 PM UTC+2, Iñaki Baz Castillo wrote:

2014-09-30 13:52 GMT+02:00 Bert Belder:
> next iteration of libuv would only support one-shot timers with an API
> similar to this:
> `uv_timeout(uv_timeout_req_t* timeout, int64_t timeout, uv_timeout_cb
> callback);

While that is great it also completely drops the "periodic timer"
feature from libuv. Ok, current periodic timers are wrong but, should
libuv provide correct periodic timers (those in which the callback
execution time does not affect the next execution)?

By just providing the uv_timeout() request, users cannot implement
real periodic timers.

Building real periodic timers isn't that hard, the user can look at uv_now() or uv_hrtime() and compute the difference between the actual time and the scheduled time for the callback. With that information one can compensate for being late, and schedule the next callback a bit earlier.

We could make it a little bit more convenient by having an API that schedules a callback on for an absolute loop time (instead of a relative one).

Note that you can already *get* the time very accurately with uv_hrtime(). The missing piece (?) is scheduling a timer with high precision.

But I can't really think of any valid use case for this, which makes it hard to tell whether any api is actually useful.
I am not so much in favor of adding features that nobody really needs (yet), which is why I'm asking for the use case the TS needs it for or has in mind.

- Bert

[Jean-Christian de Rivaz]

Le mardi 30 septembre 2014 17:26:10 UTC+2, Bert Belder a écrit :

But I can't really think of any valid use case for this, which makes it hard to tell whether any api is actually useful.
I am not so much in favor of adding features that nobody really needs (yet), which is why I'm asking for the use case the TS needs it for or has in mind.

Sorry, I have a radical different view. I build Linux embedded system since more than 15 years and I can't remember a single one that don't use periodic timer.

Most of the times this is related to the hardware or user interface where it is expected to not have delay that depend of a callback execution time to get a smooth operation.

- Jean-Christian

[Bert Belder]

On Tuesday, September 30, 2014 6:10:23 PM UTC+2, Jean-Christian de Rivaz wrote:

Le mardi 30 septembre 2014 17:26:10 UTC+2, Bert Belder a écrit :

But I can't really think of any valid use case for this, which makes it hard to tell whether any api is actually useful.
I am not so much in favor of adding features that nobody really needs (yet), which is why I'm asking for the use case the TS needs it for or has in mind.

Sorry, I have a radical different view. I build Linux embedded system since more than 15 years and I can't remember a single one that don't use periodic timer.

Note that I was speaking of a high-precision periodic timer.
I'm saying that a sloppy one (as sloppy as we have today or slighly better) can easily be built on top of a one-shot timer.

I am curious for one these embedded systems used high-precision (periodic or not) timers.
 

Most of the times this is related to the hardware

Maybe on embedded systems, I don't know. Can you give me examples of hardware other than the display (see below for that)?
Are you trying to do something very realtime like controlling a robot?

or user interface where it is expected to not have delay that depend of a callback execution time to get a smooth operation.

I think for user interfaces 60fps is generally considered good enough.
This requires a timer with at most a (1000 / 60) ≈ 16ms interval, which is well within the range of what the current timer implementation can do.
 

[Jean-Christian de Rivaz]

Le mardi 30 septembre 2014 18:25:43 UTC+2, Bert Belder a écrit :

On Tuesday, September 30, 2014 6:10:23 PM UTC+2, Jean-Christian de Rivaz wrote:

Le mardi 30 septembre 2014 17:26:10 UTC+2, Bert Belder a écrit :

But I can't really think of any valid use case for this, which makes it hard to tell whether any api is actually useful.
I am not so much in favor of adding features that nobody really needs (yet), which is why I'm asking for the use case the TS needs it for or has in mind.

Sorry, I have a radical different view. I build Linux embedded system since more than 15 years and I can't remember a single one that don't use periodic timer.

Note that I was speaking of a high-precision periodic timer.
I'm saying that a sloppy one (as sloppy as we have today or slighly better) can easily be built on top of a one-shot timer.

Ok. The question is still there about an API for a good (not high precision) periodic timer.

 

I am curious for one these embedded systems used high-precision (periodic or not) timers.
 

Most of the times this is related to the hardware

Maybe on embedded systems, I don't know. Can you give me examples of hardware other than the display (see below for that)?
Are you trying to do something very realtime like controlling a robot?

I have described my application a few post before. This one specifically require low jitter and correct phase. Most others applications I have see are not so specific, but still require relatively good periodic timer. A simple example is a cursor that can be pressed to repeat the direction of a selection but the processing to display each selection vary widely. An other simple example is a collection of devices that sample values once per second, regardless of the work need to do the sampling that could change depending of the configuration. Processing of regulation or filtering can be simpler if the input data are sampled with a good periodicity.If you think about it, you rarely expect that a regular physical action depend on a unknown delay deeply into the code execution. 

 

or user interface where it is expected to not have delay that depend of a callback execution time to get a smooth operation.

I think for user interfaces 60fps is generally considered good enough.
This requires a timer with at most a (1000 / 60) ≈ 16ms interval, which is well within the range of what the current timer implementation can do.

No. The current implementation have no way to grant any periodicity as it is completely dependent of the callback time execution. See my initial post or try it.

- Jean-Christian