Mutex contention in sql.Close()

[Joshua Garnett]

Hi All,

Has anyone else observed significant mutex contention in sql.Close() at scale?  I haven't had much success working around it beyond spinning up additional nodes.

App details

• Go 1.21.4
  
• debian:bookworm container
• App is running on an instance with 18 physical cores (36 vCPU) and 72GB of memory (only main process running)
• Connecting to PostgreSQL 15.3 AWS Aurora
• MaxOpenConns set to 125
  

Here is a screenshot of the pprof:

Thanks in advance!

--Josh

[robert engels]

It is contending with sql.rows.awaitDone().

My guess is that when you close rows it has to wait for db acknowledgement - that this is a false positive based on wall time and not cpu time - that the mutex is waiting for the cleanup operation to complete.

On Dec 13, 2023, at 1:25 PM, Joshua Garnett <josh.g...@gmail.com> wrote:

Hi All,

Has anyone else observed significant mutex contention in sql.Close() at scale?  I haven't had much success working around it beyond spinning up additional nodes.

App details

• Go 1.21.4
  
• debian:bookworm container
• App is running on an instance with 18 physical cores (36 vCPU) and 72GB of memory (only main process running)
• Connecting to PostgreSQL 15.3 AWS Aurora
• MaxOpenConns set to 125
  

Here is a screenshot of the pprof:

<Screenshot 2023-12-13 at 2.08.47 PM.png>

Thanks in advance!

--Josh

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAJAXYdi2A-rJ9YhbJJB0a1TE_MbrHL2rMOSKP1Rb%3DxHc9dOveQ%40mail.gmail.com.

[Joshua Garnett]

Hi Robert,

That makes sense.  One other oddity I've noticed is that the database CPU actually goes down when I spin up more nodes, for the same number of users/queries per second.

--Josh