Getting "runtime/cgo: pthread_create failed: Resource temporarily unavailable" crashes

[jona...@similarweb.com]

Hey,

We have a daemon that keeps crashing with the following error:

runtime/cgo: pthread_create failed: Resource temporarily unavailable

SIGABRT: abort

PC=0x393f432625

[goroutines....]

I've looked around, and it seems that this happens when using cgo (http://grokbase.com/t/gg/golang-nuts/151wh1nnnp/go-nuts-dealing-with-a-crash-in-production-of-runtime-cgo-pthread-create-failed-resource-temporarily-unavailable-with-our-new-proxy-daemon).

We have 2 XML libraries we're using in that daemon (https://bitbucket.org/martende/goxml and https://github.com/moovweb/gokogiri), both use LibXML, which is written in C, so I guess the cgo calls are coming from here.

There is a need for both libraries in our daemon - well, we need to parse XML and execute XPaths, and parse unformed HTML documents to DOM - and these libraries allow this.

Is there any way to handle such errors without changing our XML frameworks?

Thanks.

[Dave Cheney]

Calls to C via cgo are effectively a syscall, they block an entire system thread. If you cannot avoid those cgo dependencies then you will have to gate calls to those libraries with some kind of semaphore, probably using a channel as a semaphore.