Parallel processing multiple connections with sidekiq workers

115 views
Skip to first unread message

Subuddh Parekh

unread,
Oct 8, 2014, 5:24:12 PM10/8/14
to rails-sqlse...@googlegroups.com
Hi all, 

Thanks for having this group be active! 

I have a rails app that uses sidekiq workers to connect to external SQLServer databases.  Each worker connects to one external SQLServer.  I am observing that the requests are getting serialized? i.e. If a worker is connecting/working other workers (that are connecting to different Sqlservers) have to wait until the first worker finishes and then they can connect and do their work.  Is this expected behavior and then is there a way to have the connections and work(since they are all going to different servers) happen in parallel?

Thanks!
Sub

Subuddh Parekh

unread,
Oct 13, 2014, 5:51:43 PM10/13/14
to rails-sqlse...@googlegroups.com
Hi,

If this group is still active, I saw there were concerns raised a while back about parallel processing and some fixes put in to prevent the freetds calls to be non-blocking in result.c (of tiny_tds). The issue I am seeing is in the connect - if one thread(worker) is stuck in the connect, no other workers can connect (even though they are connecting to different databservers). Is there a similar fix for connect where the freetds calls are made in a non-blocking way?

Thanks
Reply all
Reply to author
Forward
0 new messages