What is the impact of allocation costs on latency?

[doo...@gmail.com]

I'm considering Go for a new service application and I'd like to minimize the time between a request being received and the response being sent (i.e., response latency). I like what I've read about all the work that's gone into the golang garbage collector to reduce stop-the-world pause times, but I've some concerns about the allocator performance. As I understand it, the Go collector never moves objects and so its allocator cannot use something like TLABs in Java (which support very fast allocation). From what I can tell, the Go allocator's fast path can only happen if there is a swept span with free objects. If the allocator needs to get a new span, it might end up having to sweep multiple spans until it encounters a span with free objects on it. This, to me, seems like a latent form of latency! I appreciate that the collector's STW events will be very short (proportional to the root set size), but I'm worried that my goroutine might try to allocate something and find itself sweeping an indeterminate number of spans before returning to service a request. Is there no upper bound on allocation time? or is this just not a problem in practice?

Thank you

[Ian Lance Taylor]

On Wed, Jun 24, 2020 at 5:28 PM <doo...@gmail.com> wrote:
>
> I'm considering Go for a new service application and I'd like to minimize the time between a request being received and the response being sent (i.e., response latency). I like what I've read about all the work that's gone into the golang garbage collector to reduce stop-the-world pause times, but I've some concerns about the allocator performance. As I understand it, the Go collector never moves objects and so its allocator cannot use something like TLABs in Java (which support very fast allocation). From what I can tell, the Go allocator's fast path can only happen if there is a swept span with free objects. If the allocator needs to get a new span, it might end up having to sweep multiple spans until it encounters a span with free objects on it. This, to me, seems like a latent form of latency! I appreciate that the collector's STW events will be very short (proportional to the root set size), but I'm worried that my goroutine might try to allocate something and find itself sweeping an indeterminate number of spans before returning to service a request. Is there no upper bound on allocation time? or is this just not a problem in practice?

That doesn't sound like an accurate description of how the memory
allocator behaves. It doesn't sweep an arbitrary number of spans. In
the current implementation, on the slow path, it looks through a
limited number of spans to find one that has space, and then it sweeps
that one and uses it. It checks up to 100 spans looking for one to
sweep; if it doesn't find any in the first 100 that it checks, it
allocates a new span. So, yes, there is an upper bound on allocation
time.

Pretty much every time that the garbage collection developers find a
place where there is unbounded latency in the memory allocator or the
GC pause time, they fix it. It's hard to be completely sure that
there aren't any left, but if there are any they are very unusual
cases.

Ian

[doo...@gmail.com]

Thank you for your reply Ian. I should have mentioned that I was looking at the sources for Go 1.14.2. I see the scheme you're describing is in the as yet unreleased 1.15 sources (and the function I was concerned about is now called 'oldCacheSpan'). It's great to see the developers drive down sources of latency like this.

[Ian Lance Taylor]

On Thu, Jun 25, 2020 at 10:32 AM <doo...@gmail.com> wrote:
>
> Thank you for your reply Ian. I should have mentioned that I was looking at the sources for Go 1.14.2. I see the scheme you're describing is in the as yet unreleased 1.15 sources (and the function I was concerned about is now called 'oldCacheSpan'). It's great to see the developers drive down sources of latency like this.

Thanks, I should have checked how new the current code was.

Ian

> On Wednesday, June 24, 2020 at 5:48:08 PM UTC-7, Ian Lance Taylor wrote:
>>
>> On Wed, Jun 24, 2020 at 5:28 PM <doo...@gmail.com> wrote:
>> >
>> > I'm considering Go for a new service application and I'd like to minimize the time between a request being received and the response being sent (i.e., response latency). I like what I've read about all the work that's gone into the golang garbage collector to reduce stop-the-world pause times, but I've some concerns about the allocator performance. As I understand it, the Go collector never moves objects and so its allocator cannot use something like TLABs in Java (which support very fast allocation). From what I can tell, the Go allocator's fast path can only happen if there is a swept span with free objects. If the allocator needs to get a new span, it might end up having to sweep multiple spans until it encounters a span with free objects on it. This, to me, seems like a latent form of latency! I appreciate that the collector's STW events will be very short (proportional to the root set size), but I'm worried that my goroutine might try to allocate something and find itself sweeping an indeterminate number of spans before returning to service a request. Is there no upper bound on allocation time? or is this just not a problem in practice?
>>
>> That doesn't sound like an accurate description of how the memory
>> allocator behaves. It doesn't sweep an arbitrary number of spans. In
>> the current implementation, on the slow path, it looks through a
>> limited number of spans to find one that has space, and then it sweeps
>> that one and uses it. It checks up to 100 spans looking for one to
>> sweep; if it doesn't find any in the first 100 that it checks, it
>> allocates a new span. So, yes, there is an upper bound on allocation
>> time.
>>
>> Pretty much every time that the garbage collection developers find a
>> place where there is unbounded latency in the memory allocator or the
>> GC pause time, they fix it. It's hard to be completely sure that
>> there aren't any left, but if there are any they are very unusual
>> cases.
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6947c21d-439e-483b-a0e0-3e621cf3e7bao%40googlegroups.com.