Lee Ryall
unread,Jan 10, 2023, 3:35:30 PM1/10/23Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Sign in to report message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Salt-users
Hey,
I am having a problem with my minions when I rebuild my salt master (maintaining keys etc...)
After the master has been rebuilt the minion will consistently throw tornado errors into the log.
If I initiate a salt-call state.apply from the minion machine it registers in salt-key on the master but the state.apply exceptions out.
If I restart the minion service then all is right with the world again.What is it that I need to look at to get the minion to reconnect without intervention?
Log/exception is below...
2023-01-06 15:42:13,489 [tornado.application:353 ][ERROR ][615689] Future <salt.ext.tornado.concurrent.Future object at 0x7fbd02ddf280> exception was never retrieved: Traceback (most recent call last):
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/minion.py", line 2697, in handle_event
yield _minion.req_channel.send(
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/channel/client.py", line 295, in send
ret = yield self._crypted_transfer(load, timeout=timeout, raw=raw)
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/channel/client.py", line 252, in _crypted_transfer
ret = yield _do_transfer()
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/channel/client.py", line 233, in _do_transfer
data = yield self.transport.send(
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/transport/zeromq.py", line 914, in send
ret = yield self.message_client.send(load, timeout=timeout)
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "salt/ext/tornado/gen.py", line 1064, in run
yielded = self.gen.throw(*exc_info)
File "salt/transport/zeromq.py", line 624, in send
recv = yield future
File "salt/ext/tornado/gen.py", line 1056, in run
value = future.result()
File "salt/ext/tornado/concurrent.py", line 249, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
salt.exceptions.SaltReqTimeoutError: Message timed out