100% CPU on load of my go application

[Gavra]

Hi,

We encountered an issue where our application consumed 100% CPU. The app was stuck (instruction-wise). It is an app we run in a docker alpine container.

The execve is our app. This happens right when we start it.

The NULL param for mmap and the large buff size for getaffinity got my eye but we have no idea how to proceed. It appears this happens prior to any logic of our app.

Any idea?

Thanks!

[Robert Engels]

I would start by testing on the latest Go version. You really haven’t provided anywhere enough details to help. I suspect if your app is not starting at all you have a bad image. 

On Feb 6, 2025, at 7:32 AM, Gavra <gav...@gmail.com> wrote:

Hi,

We encountered an issue where our application consumed 100% CPU. The app was stuck (instruction-wise). It is an app we run in a docker alpine container.

The execve is our app. This happens right when we start it.

The NULL param for mmap and the large buff size for getaffinity got my eye but we have no idea how to proceed. It appears this happens prior to any logic of our app.

Any idea?

Thanks!

<image (2).png>

<Screenshot 2025-02-06 at 10.15.27.png>

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/724f4958-2fb4-4f08-ac25-fd861393f9a8n%40googlegroups.com.

<Screenshot 2025-02-06 at 10.15.27.png>

<image (2).png>

[Sharon Mafgaoker]

Can you try to start it locally on your machine?

Sharon Mafgaoker – Senior Solutions Architect 

M. 050 995 99 16 | sha...@cloud5.co.il

To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/3E08FAFE-4F65-4A7C-B3CB-DA2A456D068C%40ix.netcom.com.

[Elad Gavra]

We run a simple go program and we see the same syscalls so that really provides nothing. I think we will keep an eye on it and understand it better.

[David Finkel]

On Thu, Feb 6, 2025 at 9:38 AM Elad Gavra <gav...@gmail.com> wrote:

We run a simple go program and we see the same syscalls so that really provides nothing. I think we will keep an eye on it and understand it better.

The main benefit to running the binary outside a container is that you get more flexibility for debugging. (especially toolwise)

My recommendation would be to add an HTTP server with net/http/pprof linked in so you can grab CPU profiles and goroutine dumps.

https://pkg.go.dev/net/http/pprof#pkg-overview

I don't remember how well it works for Go binaries' debug info, but you can use the perf tool to get profiling information both system-wide and per-process. (pprof is usually quite good, so I don't usually switch to perf)

 

On Thu, Feb 6, 2025, 16:35 Sharon Mafgaoker <sha...@cloud5.co.il> wrote:

Can you try to start it locally on your machine?

Sharon Mafgaoker – Senior Solutions Architect 

M. 050 995 99 16 | sha...@cloud5.co.il

On Thu, 6 Feb 2025 at 16:27 Robert Engels <ren...@ix.netcom.com> wrote:

I would start by testing on the latest Go version. You really haven’t provided anywhere enough details to help. I suspect if your app is not starting at all you have a bad image. 

On Feb 6, 2025, at 7:32 AM, Gavra <gav...@gmail.com> wrote:

Hi,

We encountered an issue where our application consumed 100% CPU. The app was stuck (instruction-wise). It is an app we run in a docker alpine container.

The execve is our app. This happens right when we start it.

The NULL param for mmap and the large buff size for getaffinity got my eye but we have no idea how to proceed. It appears this happens prior to any logic of our app.

That first NULL parameter to mmap is normal. It's asking the kernel to pick a random address (in this case, for Go's heap)

The sched_getaffinity call is used by the Go runtime to compute how many cores are available and default GOMAXPROCS to something reasonable.

Any idea?

Thanks!

<image (2).png>

<Screenshot 2025-02-06 at 10.15.27.png>

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/724f4958-2fb4-4f08-ac25-fd861393f9a8n%40googlegroups.com.

<Screenshot 2025-02-06 at 10.15.27.png>

<image (2).png>

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/3E08FAFE-4F65-4A7C-B3CB-DA2A456D068C%40ix.netcom.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/CAMwNhAkX%2BZ%2BDKopCm13ZWgWXhtFvm_3OjKC5dOOWXUMtbHk2eg%40mail.gmail.com.