[WARNING ] Skipped invalid cache mtime entry in /var/cache/salt/master/roots/mtime_map
Markus Kramer
In /var/log/salt/master we finally found hundreds of entries in the form
<time> [salt.loaded.int.fileserver.roots ][WARNING ] Skipped invalid cache mtime entry in /var/cache/salt/master/roots/mtime_map: /srv/salt/foo/bar/filename
Many file names
- are truncated, sometimes to one character, e.g. /
- contain thousands of "characters" represented as ^@ (caret at-sign),
The value of ^@ is 0 (zero), according to `od` (octal dump) .
Restarting the master seems to "cure" the master.
Has anybody seen this before?
What happened?
We need to regularly update files in /srv/salt/ because we want to distribute binary files smaller than 10 kbyte to the minions.
Is there a necessary step before or after copying a file to /srv/salt/ ?
Salt-master is version 2014.7.4 on Debian.
Markus Kramer
Colton Myers
--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
markus...@posteo.de
Thank you for confirming that there are no necessary steps before modifying /srv/salt, but is this also true for orchestration/reactor?
In an immediate reaction to the problem we changed three things:
- We modified /srv/salt only after stopping the salt-master.
Since a few days I modified /srv/salt with a running master and have not observed any mtime_map problems, which is good and consistent with your comment. - We no longer copy recursively ~ 500 files (the whole svn repository) to /srv/salt but only those ~ 10 small files which we actually want to `file.manage`
- We no longer modify /srv/salt from orchestration/reactor (but we need to do that again or find an alternative)
We modified /srv/salt with an orchestrated reactor, which Seth House was so kind to write as a a working example
In Seth' code, though, I changed
{% set db_file = '/tmp/job_events.sqlite3' %}
to
{% set db_file = '/srv/salt/job_events.sqlite3' %}
and added (in the orchestration sls) the creation of files in /srv/salt
To our great horror we observed many hundreds of ^@ characters within the file names of mtime_map.
As soon as we put the modifications outside /srv/salt, the repeated characters ^@ (character 0) disappeared.
Unfortunately, this is no long term solution for us. In the near future we will need to modify /srv/salt from orchestration/reactor:
one minion regularly sends data (~ 60 byte) to the master, the master stores it and sends it to all other minions.
The ideal format for the data is a directory of small files, which is kept in sync between the master and the minions.
Is there a way to do that?
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/UhlhMZLxsqw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
Stephen Spencer
Set up another salt-master on another host. I am using a similar pattern with a runner updating x509 CRL files under /srv/salt without the need for stopping the master. Also, what os/distribution are you using for your master? I'm using Cent7.
-S
Stephen Spencer
(sorry.. incomplete thought there)
set up another master with some. test minions to see if it might not be an issue with a corrupted 3rd party library.
-S
markus...@posteo.de
I am not familiar with 'runner'. Can you sketch your solution, please?
My master runs on Debian, and setting up a scond master is on my my todo list.
CentOs would make sense because our data centre runs RedHat, but we will finish the proof of concept on Debian.