Stuck getting started with slack
Mike
but I'm stuck and it's frustrating because I feel like I'm close.
Here's what I'm doing and how I assume slack works:
(below is based on slack version 0.15.2)
I have three linux machines - master1, slave1, slave2 - where master1
has slack installed and slave1 and slave2 are configured with SSH keys
so master1 can login to each without prompting for a password. What I
would like to do is maintain /etc/foo.conf across the two slave
machines by editing it on master1 and syncing with the two slaves.
Master1 is configured as such:
- /etc/slack.conf:
ROLE_LIST=etc/roles.conf
SOURCE=/var/lib/slack/repo
CACHE=/var/cache/slack
STAGE=/var/lib/slack/stage
ROOT=/
BACKUP_DIR=/var/lib/slack/backups
- /var/lib/slack/repo/etc/roles.conf
slave1: role1
slave2: role1
- /var/lib/slack/repo/roles/role1/files/etc/foo.conf
"this is a test"
If I run slack -vvv with this configuration, I get the following
error:
root@master1:/var/lib/slack/repo# slack -vv
slack: getroles
slack-getroles: Reading '/var/lib/slack/repo/etc/roles.conf'
FATAL[slack-getroles]: Host 'master1' not found in '/var/lib/slack/
repo/etc/roles.conf'!
FATAL[slack]: '/usr/lib/slack/slack-getroles --verbose' exited 255 at /
usr/sbin/slack line 205
If I add master1 to $SOURCE/etc/roles.conf, it correctly syncs
foo.conf to master1, but it never makes it to slave1 or slave2:
root@master1:/var/lib/slack/repo# slack -vv
slack: getroles
slack-getroles: Reading '/var/lib/slack/repo/etc/roles.conf'
slack: installing roles: role1
slack: sync role1
slack-sync: Checking perms on '/var/cache/slack/roles/'
slack-sync: Syncing cache with 'rsync --cvs-exclude --recursive --copy-
links --times --perms --sparse --delete --files-from=- --from0 /var/
lib/slack/repo/roles/ /var/cache/slack/roles/'
slack: stage files role1
slack-stage: Checking perms on '/var/lib/slack/stage/roles'
slack-stage: Syncing role1/files with 'rsync --recursive --times --
ignore-times --perms --sparse --delete /var/cache/slack/roles/role1/
files/ /var/lib/slack/stage/roles/role1/files/'
slack-stage: Setting default perms on /var/lib/slack/stage/roles/role1/
files
slack: stage scripts role1
slack-stage: Checking perms on '/var/lib/slack/stage/roles'
slack-stage: Syncing role1/scripts with 'rsync --recursive --times --
ignore-times --perms --sparse --delete /var/cache/slack/roles/role1/
scripts/ /var/lib/slack/stage/roles/role1/scripts/'
slack-stage: Setting default perms on /var/lib/slack/stage/roles/role1/
scripts
slack: preinstall role1
slack-runscript: Skipping '/var/lib/slack/stage/roles/role1/scripts/
preinstall' because it doesn't exist
slack: fixfiles role1
slack-runscript: Skipping '/var/lib/slack/stage/roles/role1/scripts/
fixfiles' because it doesn't exist
slack: install role1
slack-installfiles: Creating backup directory '/var/lib/slack/backups/
2008-10-07-00:15:17'
slack-installfiles: Syncing '/var/lib/slack/stage/roles/role1/files/'
to '//'
slack: postinstall role1
slack-runscript: Skipping '/var/lib/slack/stage/roles/role1/scripts/
postinstall' because it doesn't exist
Am I misunderstanding slack at a fundamental level? It seems like it
should be able to rsync foo.conf to the slave servers, but it never
does.
Any help would be sincerely appreciated.
Thanks,
Mike
Steve Atwell
It sounds like the bit of info that you're missing is that slack is a
pull-based system, not a push-based system. The basic idea behind
slack is that you have a central repository of roles. Then on each
machine you want to install a role on, you run slack. Running slack
fetches the roles.conf file and the necessary roles from the central
repository and installs the roles configured for the current host.
So in your case, you'd want to install slack on slave1 and slave2, and
then configure slack.conf on them to fetch from master1. E.g., set
SOURCE to master1:/var/lib/slack/repo to pull via ssh. (See the
slack.conf man page for other methods of fetching the repository.)
Then you connect to a host (e.g., slave1) and run slack. Slack will
use SOURCE to determine that it will connect to master1 via SSH and
fetch and install what it needs from there.
Mike
> It sounds like the bit of info that you're missing is that slack is a
> pull-based system, not a push-based system.
model is insinuated in the documentation, I just wasn't seeing it. Of
course now that I know this it sticks out like a sore thumb.
> So in your case, you'd want to install slack on slave1 and slave2, and
> then configure slack.conf on them to fetch from master1. E.g., set
> SOURCE to master1:/var/lib/slack/repo to pull via ssh.
academic question, not trying to denigrate slack - why pull over
push? It seems to me that every *nix out there has SSH, and
especially given that rsync is the transport method, why not exploit
that rather than installing and maintaining a new package all over
your network? Thankfully, it's perl instead of C or ruby, but still,
it seems like an unnecessary burden when you could easily set up a
central server with all of your roles, commit them to your VCS of
choice and combining that with the use sudo, actually see who made
changes to system configurations. I dunno, just a thought. Maybe
worthy of a slack spinoff. :)
Mike
Philip Walls
Using slack does not preclude you from storing your roles in version control. I would imagine most people on this list are doing just that.
To give an example, I have a bzr repository with all of my slack configs which gets synced onto my slack master every 15 minutes. All of my hosts are configured to slack against that master server.
There are several complications with a push based model which is why many configuration management systems (including slack) use pull:
- You must deal gracefully with offline hosts (and retry until they are up)
- Your master server must be able to automatically authenticate to every host in your fleet
- If individual hosts should receive updates at a different time of day or several times a day, you must cope with this on the master
- If you want to get configuration updated immediately on a certain host, you must trigger a push on the master
That said, slack and push are not mutually exclusive. It is possible to trigger a local slack run over SSH (or any other protocol), but you'd still need to have slack installed on the slave host. It's not clear that there is any benefit from this method, as it suffers from several of the limitations I mentioned above and still requires having slack installed locally.
-P
On Tue, Oct 7, 2008 at 4:58 AM, Mike wrote:
Alan Sundell
> Ok, so I definitely get it now, but I wonder - and this is purely an
> academic question, not trying to denigrate slack - why pull over
> push?
No problem...
It's pull instead of push for all the standard reasons -- if you have
a bunch of hosts, some of them are going to be down at any given time.
A push master would have to keep track of state for all the hosts,
manage scheduling, and so on. There are a lot of cases that can cause
a push master to get bogged down, such as some hosts being on a slow
WAN link. Putting all this intelligence on the master makes things
complicated (what if you have one master shared between groups that
want to manager their own private push schedules?), and you have to
code for all that. It's much easier to delegate management of all
this to the clients.
That said, slack can work perfectly well in a push setup. All you
have to do is turn off syncing (--no-sync or NO_SYNC=1 in slack.conf)
and slack won't try to populate the CACHE, so you can populate it
yourself with whatever means you want. This is handy in certain
situations, but I wouldn't recommend it for normal practice.
I've seen --no-sync used to:
1) get around DMZ firewalls that block outbound connections
2) separate the install schedule from the sync schedule, to deal with
irresponsible people who have root -- you can overwrite config files
from the CACHE every few minutes to keep people from editing locally,
but only push from SOURCE when you're ready to release (obviously, you
can still use pull here, and it's nicer to re-educate the
irresponsible people, but everyone has their own unique constraints)
> It seems to me that every *nix out there has SSH, and
> especially given that rsync is the transport method, why not exploit
> that rather than installing and maintaining a new package all over
> your network?
Take a look at the code in slack-installfiles. We have to have some
code with access to the local filesystem to be able to make new
directories when needed but avoid setting permissions on existing
directories (e.g. /usr/local). If we had some sort of manifest file,
we could avoid this, since people could tell us explicitly what they
intend instead of us having to figure it out implicitly, but making
people generate manifest files would kind of defeat the point of
slack.
Plus, of course, we want to run a bunch of scripts, do it in a clean
environment, etc.
Having the code installed on the client instead of scp-ed over or
something is just a lot easier to deal with. It also lets you test
newer versions of the code more easily (just install the new package
in one place).
Anyway, installing a package everywhere isn't as hard as it used to
be, thanks to things like APT. If you have enough hosts where this is
a serious problem, you're gonna want to look at automating your
installs with stuff like PXE, anyway, and you can add the package to
your automated install.
One of the core assumptions driving slack is that reinstalls are easy
(that's why it doesn't support removing files, or uninstalling roles,
etc). The idea is you use PXE or something to install your base OS,
and slack on top of that to configure it. If you need to re-use the
machine for something else, just wipe and reinstall. Making that kind
of operation easy is the whole point of automating things, and slack
can be used as a piece of that equation. Some people use it to
configure just the base OS (e.g. nsswitch.conf) and have something
else to run their services; others go the other way around (e.g. a
webserver role). Of course, slack doesn't really care -- it's just a
dump script to move files around, but the intent behind it is to be
part of automating builds and changes.
> Thankfully, it's perl instead of C or ruby, but still,
> it seems like an unnecessary burden when you could easily set up a
> central server with all of your roles, commit them to your VCS of
> choice and combining that with the use sudo, actually see who made
> changes to system configurations. I dunno, just a thought. Maybe
> worthy of a slack spinoff. :)
Ah, I see Philip has responded. To echo him, yes, you absolutely can
(and should) put your roles in a VCS of your choice, and lots of
people run slack under sudo (that's why slack-runscript clears the
environment, so you won't be dependent on something leaked through
from the user environment by sudo).
Hope that answers your questions :)
--Alan
Mike
candid and well-informed. I am curious, though, when it comes to the
'pull' architecture how you kick-off a mass pull? Say for example you
have 1000 servers who need to have /etc/resolv.conf updated because a
new DNS server was added. Is cron the the only sane way to
accomplish this? Does anyone do something like SSH into all 1000
machines and kick off a `slack role-resolv-conf`? What if there was
an emergency where you needed to update that file before cron would
kick off a sync?
As for the push vs. pull debate, I still think that I'd rather try to
workaround the issues with push since they are less relevant in my
particular environment (no WAN issues, every machine *should* be up,
etc.), but it's great to know what others are doing. If my way
doesn't work, at least I have another option. :)
Thanks to all for the help!