Issues when using time.Ticker microsecond duration

[envee]

Hi All,

I understand this issue has been discussed in the past, and I have seen a few comments from Ian and Russ Cox about this topic, but I could not arrive at a conclusion about how I can make the time.Ticker send me ticks at atleast close to the Ticker duration I specify. At the moment, I am seeing ticks being sent at way over the specified duration especially when I have sub-millisecond durations. 

With 1ms duration, the ticker seems to be quite close to the duration (maybe within +/-5%). I would be happier if it were within 1%, but I can handle that.

With 1 micro-second duration, the ticker sends ticks nearly 4 times slower than expected.

Here is my sample code

"

ti := time.NewTicker(1 * time.Millisecond)

start := time.Now()

for i := 0; i < 1000; i++ {

<-ti.C

}

fmt.Printf("elapsed time = %v\n", time.Since(start))

"

The output is usually close to 1 second as expected (close to) when using 1ms duration.

elapsed time = 1.000159338s

My code for the microsecond test is :

"

ti := time.NewTicker(1 * time.Microsecond)

start := time.Now()

for i := 0; i < 1000000; i++ {

<-ti.C

}

fmt.Printf("elapsed time = %v\n", time.Since(start))

"

With this, I get the following output which shows the elapsed time close to 4 seconds :

elapsed time = 4.796662856s

Is there anyway, I can ensure ticks are sent at approximately within 5% of my specified duration ?

I understand from time.Ticker docs that ticks will be lost if the ticker handler cannot keep up with the ticker time.

In the simple code above, I am literally doing very minimal work in each loop iteration and that is about checking the loop condition.

So I am not sure why ticks are being lost ?

[Robert Engels]

See https://github.com/golang/go/issues/27707

On Jan 30, 2022, at 5:50 PM, envee <neeraj....@gmail.com> wrote:

Hi All,

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/5f55ccf9-1478-4490-a8f4-a35a5764721dn%40googlegroups.com.

[envee]

Thanks, but I am not sure how that reference solves the issue ?

Or are you suggesting that the only way is to use Cgo and invoke usleep to get very close to the Ticker duration specified ?

[Kurtis Rader]

On Sun, Jan 30, 2022 at 7:16 PM envee <neeraj....@gmail.com> wrote:

Thanks, but I am not sure how that reference solves the issue ?

Or are you suggesting that the only way is to use Cgo and invoke usleep to get very close to the Ticker duration specified ?

On Monday, 31 January 2022 at 11:25:58 UTC+11 ren...@ix.netcom.com wrote:

See https://github.com/golang/go/issues/27707

The reference to that issue does not solve your problem. It illustrates that this issue has been noted, and discussed, for at least three years. It hasn't been resolved because a solution is difficult. It would help if you provided a concrete example why this matters. As has been done in the referenced issue.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Robert Engels]

Pretty much what Kurtis said. Low interval timers usually require specialized constructs if not a real time OS to efficiently (or even at all). 

On Jan 30, 2022, at 9:16 PM, envee <neeraj....@gmail.com> wrote:

Thanks, but I am not sure how that reference solves the issue ?

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/efa204c2-092f-46e1-abe4-8aa5c502d986n%40googlegroups.com.

[envee]

Thanks Kurtis and Robert.

My use-case is for a telecommunications application (HTTP/2 client, 5G client to be precise) which needs to send 5G (HTTP/2) requests to a server at a configurable rate (TPS). 

While the telecom industry very commonly use the word TPS (Transactions Per Second), I would like to control the rate within the second i.e. if the TPS is to be 2000, then I would like to spread these 2000 transactions (HTTP/2 requests) over 1 second uniformly i.e. 1 request every 500 mico-seconds which will then result in 2000 requests in that 1 second.

I believed that by using a time.Ticker to fire every 500 micro-seconds, I would be able to achieve this outcome.

Is there any other way you could suggest I do this using Go ? 

[Robert Engels]

If you need a “guarantee” in these efforts it is much more complex. You must likely need a real-time OS and interrupt driven code. A general purpose language, os and hardware probably won’t cut it. 

This of it this way though, if you had 20 cpus to spread the work on, you only need to interrupt every 10 ms - far more achievable. As long as the work is independent. 

On Jan 30, 2022, at 11:10 PM, envee <neeraj....@gmail.com> wrote:

Thanks Kurtis and Robert.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3f9d1c36-4820-4555-a5f3-9721e661530bn%40googlegroups.com.

[Robert Engels]

Sorry for typos. Early morning on small phone with no coffee and old person eyes. :)

On Jan 31, 2022, at 7:13 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[Amnon]

Irrespective of the language, these sort of delays are always at the mercy of the scheduling jitter of the underlying operating system.
The operating system is at liberty to preempt any application at any time, and then resume it some arbitrary time later. If you ask the OS to sleep for 
1ms, the OS will deschedule the process, and then resume at some point in time after 1ms. Exactly how long after depends 
on what else is running on the processor, contention for cores etc. Once the process gets to run again, the Go runtime 
does its own scheduling, it may have many more goroutines ready to run than it has available OS threads. So the gorountine which 
is waiting on the ticker could wait an arbitrary amount of time beyond the minimum 1ms delay.

The demand for all delays to be within 5% of 1ms is hard to achieve in any language running on top of a general purpose (non-realtime) OS,  and is double hard in Go which adds an additional layer of scheduling.

A better approach would be to start with a 1ms ticker, but then at each tick calculate the amount of time which has actually elapsed, and then send the required number of requests to catch-up with the required request rate. This will mean a slightly uneven and jittery request rate at the microscopic level, and the amount of jitter will be affected by how heavily loaded the machine is. But the request rate will even out when looking over longer time periods. This will be the best you can do in Go, or in user-space in any language running on a non-real-time OS. 

[Uli Kunitz]

Please have a look at golang.org/x/time/rate. This package should solve your problem.

Here is an example with 10 events per second.

func main() {
        log.SetFlags(log.Lmicroseconds)

        l := rate.NewLimiter(10, 1)
        ctx := context.Background()
        for {
                if err := l.Wait(ctx); err != nil {
                        log.Fatal(err)
                }
                log.Print("event!")
        }
}

The package is intended to rate external events, but the simple loop above works as well. You can generate/handle events also in parallel threads if  one CPU is not sufficient. Note that for high frequencies it doesn't ensure that the time intervals are exact, but it ensures that the rate limit is maintained.

[Ian Davis]

On Mon, 31 Jan 2022, at 8:33 PM, Uli Kunitz wrote:

Please have a look at golang.org/x/time/rate. This package should solve your problem.

Here is an example with 10 events per second.

func main() {
        log.SetFlags(log.Lmicroseconds)

        l := rate.NewLimiter(10, 1)
        ctx := context.Background()
        for {
                if err := l.Wait(ctx); err != nil {
                        log.Fatal(err)
                }
                log.Print("event!")
        }
}

The package is intended to rate external events, but the simple loop above works as well. You can generate/handle events also in parallel threads if  one CPU is not sufficient. Note that for high frequencies it doesn't ensure that the time intervals are exact, but it ensures that the rate limit is maintained.

I agree that this is the right solution for the OP task, but note that the rate package is also affected by the same timer issues: https://github.com/golang/go/issues/47084

[Uli Kunitz]

In my tests 2000 events per second appear to be fine, but there is an issue a magnitude larger with 20 000 events per second.

[Robert Engels]

Because 20000 is a magnitude larger than 2000.

> On Feb 1, 2022, at 1:44 PM, Uli Kunitz <uli.k...@gmail.com> wrote:
>

[envee]

Thanks Robert, Uli and Ian for your suggestions.

I think what I will probably do is use a Ticker with a duration of 1ms. At every 1ms, I will "wake-up" N number of goroutines to trigger HTTP requests.

That number N = (request rate per second / 1000)  = requests per ms.

So, if I need to ensure a rate of 10000 requests per second, I believe it should be possible for the Ticker to return after every 1ms and then fire about 10 requests at every such interval.

From the tests that I have performed, I can see that a Ticker pretty accurately fires at every 1ms interval.

I think it's only when the Ticker duration falls below 1ms, that I see issues.

If my desired rate is less than 1000 per second, then I will create a Ticker to return every 1000/request rate milliseconds, which will be a number greater than 1ms. 

This approach is closely based on Robert's suggestion about using a higher duration for Ticker time and waking up a small subset of goroutines.

I think it should be ok for a client to be accurate at the level of granularity of 1ms.

[envee]

And I forgot to mention that approach I mentioned talks about waking up N goroutines at a time. The way I plan to do is to select a range of N goroutines from my list of goroutines and only allow those goroutines to send HTTP requests.

I could use this approach to select the N goroutines or even use a Semaphore. I presume using a Semaphore, will allow a good amount of random N goroutines out of M goroutines to execute.

[Robert Engels]

I am unclear why you need to use an N of M. I would make sure the hardest case is handled and you can use a variety of techniques to partition the work. 

On Feb 2, 2022, at 6:58 PM, envee <neeraj....@gmail.com> wrote:

And I forgot to mention that approach I mentioned talks about waking up N goroutines at a time. The way I plan to do is to select a range of N goroutines from my list of goroutines and only allow those goroutines to send HTTP requests.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/209b18aa-1317-4d72-80a1-222f10a26013n%40googlegroups.com.

[Amnon BC]

> From the tests that I have performed, I can see that a Ticker pretty accurately fires at every 1ms interval.

It will depend on the load on your machine. As the load on the machine increases, so with the jitter in the tick time.

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/3GUnjdoWjqU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/F912EE7E-4F80-4C16-ABC1-943572FD576C%40ix.netcom.com.

[envee]

Uli, Robert, 

I finally settled on using the rate limiter package to achieve what I need. I have a machine with 40 vCPUs and when I use a rate limiter with a rate limit of 1000, I am able to generate HTTP requests at that rate. (There are other processes running on this RHEL 7 machine, but it is very lightly loaded).

When I increase the limit to anything higher than 3000 TPS (transactions per second), my program can only generate 1000 TPS less than that rate.

For example, if I need to send 4000 TPS, I can see (using some custom Prometheus metrics), that I am generating only 3000 TPS. Likewise, if I set the limit to 8000 TPS, I can see request rate of 7000 TPS. 

I can live with that for now. Because atleast I have a mechanism to control the rate and get close to what I need.

I will see if I can tune my program any further to get much closer to the desired rate.

Thank you all for your help.

[envee]

Hi All,

Just thought I'd let you know how I went with the rate limiter.

After several searches on Google about why rate limiter does not generate events at the rate I requested, I hit on the following issue on Github and saw this comment from Chris Hines about setting a burst value equal to the limiter limit i.e. if the desired rate is 2000, then set burst to 2000 as well.

https://github.com/golang/go/issues/47084#issuecomment-897291261

With this change to my code, I was able to get events generated at the rate I specify i.e. if I configure my application to generate 1000 transactions per second (TPS), I get 1000. If 5000, then I get 5000 etc.

It appears this is something to do with the change to the netpoller sleep interval granularity being 1ms on linux.

For now, I am able to generate transactions at the rate I specify, so I am in a much better position as of now - with no loss of TPS.