Minion unable to reconnect after rebuild

8 views
Skip to first unread message

Lee Ryall

unread,
Jan 10, 2023, 3:35:30 PM1/10/23
to Salt-users
Hey,
I am having a problem with my minions when I rebuild my salt master (maintaining keys etc...)
After the master has been rebuilt the minion will consistently throw tornado errors into the log.
If I initiate a salt-call state.apply from the minion machine it registers in salt-key on the master but the state.apply exceptions out.
If I restart the minion service then all is right with the world again.What is it that I need to look at to get the minion to reconnect without intervention?

Log/exception is below...
2023-01-06 15:42:13,489 [tornado.application:353 ][ERROR   ][615689] Future <salt.ext.tornado.concurrent.Future object at 0x7fbd02ddf280> exception was never retrieved: Traceback (most recent call last):
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/minion.py", line 2697, in handle_event
    yield _minion.req_channel.send(
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/channel/client.py", line 295, in send
    ret = yield self._crypted_transfer(load, timeout=timeout, raw=raw)
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/channel/client.py", line 252, in _crypted_transfer
    ret = yield _do_transfer()
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/channel/client.py", line 233, in _do_transfer
    data = yield self.transport.send(
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/transport/zeromq.py", line 914, in send
    ret = yield self.message_client.send(load, timeout=timeout)
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "salt/ext/tornado/gen.py", line 1064, in run
    yielded = self.gen.throw(*exc_info)
  File "salt/transport/zeromq.py", line 624, in send
    recv = yield future
  File "salt/ext/tornado/gen.py", line 1056, in run
    value = future.result()
  File "salt/ext/tornado/concurrent.py", line 249, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
salt.exceptions.SaltReqTimeoutError: Message timed out

Reply all
Reply to author
Forward
0 new messages