Hi Team
We had a misbehaving task running in a loop which we had to bring it down. While the task was running we stopped appscale. After this, when we restarted appscale, the app is not coming up.
When we examined log, one thing caught our eye. In the transaction groomer log, we are seeing this:
========
2020-03-09 01:17:46,579 ERROR transaction_groomer.py:196 Unexpected error while grooming localapp
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/appscale/datastore/scripts/transaction_groomer.py", line 192, in start
yield self._groom_project()
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 215, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/local/lib/python2.7/dist-packages/appscale/datastore/scripts/transaction_groomer.py", line 256, in _groom_project
oldest_valid_tx_time = yield self._fetch_and_clean(index, worker_count)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 215, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/local/lib/python2.7/dist-packages/appscale/datastore/scripts/transaction_groomer.py", line 381, in _fetch_and_clean
new_children = yield self._tornado_zk.get_children(container_path)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 215, in result
======
Can you let us know what could be going wrong? Any help is greatly appreciated.
How do we recover from this error?