1. Are you able to get some logs to see whether there are any errors around when the connections are dropped?
Two suspicions:
1a. There were some errors / crashes - having logs will help
1b. I am not familiar with Gearman - but perhaps there are some config that timeouts / resets a connection after some time? Again, having logs to see whether there are any clues that this may be happening will help
2. When the connections were dropped, are you running any RPCs? Are they long-running streaming RPCs? Or are we talking about the server being idle?
-Stanley