Add timeout for synchronous kazoo.client.KazooClient.get method

69 views
Skip to first unread message

Alexander Gryanko

unread,
Oct 4, 2013, 6:00:59 PM10/4/13
to pyth...@googlegroups.com
I store tasks in ZooKeeper for processing big amount of data.  Before processing I need to download data from external storage. My script creates subprocessing thread to add task for downloading data in proprietary application and when job was completed application is calling method to set property 'state' to 'c' in ZooKeeper.

For waiting data in script I've got method called wait_download. It's waiting for property 'state' == 'c' in ZooKeeper and after occurring of event script continue execution. But if application was crashed or any other problem was prevented changing property 'state', my application is hanging. 

 77     def __zk_wait(self, event):
 78         state = self.zkh.get(event.path)
 79         event_path = os.path.dirname(event.path)
 80         if event.type == 'CHANGED' and state[0] == 'c':
 81             self.logger.info('  task {job_path} has been downloaded'.format(
 82                 job_path=event_path))
 83             self.lock.release()
 84
 85     def wait_download(self, job_path):
 86         self.lock = Lock()
 87
 88         self.lock.acquire()
 89         chld = self.zkh.get(job_path + '/status', watch=self.__zk_wait)
 90         self.lock.acquire()

Is there any way to set timeout for synchronous implementation of get method? I was reading a code of KaZoo, but don't understand how it calls '_call' with GetData. None of all returned classes don't have any property to set timeout. 
Reply all
Reply to author
Forward
0 new messages