Configuration file - start lsynd
David
I'm developing an application for my job and I'm using lsyncd ver 1.26
(maybe the next questions are solved with latest vesions...).
I have to questions/problems.
1. If there are two clients using lsync to share information, the
lsyncd launch the rsync just only when detects an event in the local
folder of the client. Are there any way to sync the files, using lsyng
from the source and the target in both directions? Can it cause any
problems such a infinite loop?
2. I'm using a Ubuntu 10.2 dist. Can I launch lsynd with any user, not
with root? Because when I try to launch with other users the deamon
doesn't start (only with root).
Thanks,
David
Axel Kittenberger
> Are there any way to sync the files, using lsyng
> from the source and the target in both directions? Can it cause any
> problems such a infinite loop?
Yes it can cause problems and cause infinite loops. Generally its not
adviced to do this, you are going very much on the experimental side
of things with this. However, practically it is possible, as e.g.
Gordon here runs it that way. You will have to set rsync to use a tmp
directory and not create files in place. Still there a unwanted race
conditions if a directory changes at both sites at the same time.
> 2. I'm using a Ubuntu 10.2 dist. Can I launch lsynd with any user, not
> with root? Because when I try to launch with other users the deamon
> doesn't start (only with root).
Yes, you can do that, and I run lsyncd that way myself. You just have
to have a user which has read access to all the files you watch.
I don't know why it doesnt start in your case. Are you using a script?
Are you trying to start it by hand? Do you have log outputs?
Kind regards,
Axel
Gordan Bobic
> Hello, I'm David and I'm new in this group.
>
> I'm developing an application for my job and I'm using lsyncd ver 1.26
> (maybe the next questions are solved with latest vesions...).
>
> I have to questions/problems.
>
> 1. If there are two clients using lsync to share information, the
> lsyncd launch the rsync just only when detects an event in the local
> folder of the client. Are there any way to sync the files, using lsyng
> from the source and the target in both directions? Can it cause any
> problems such a infinite loop?
This is the way I'm running it. Have a look through the recent threads
for discussions on the subject. Be aware that there is an inherent race
condition involved in this. You may find that the 2-way sync causes new
files to get deleted before they are replicated, and you may find that
big recursive deletes don't end up fully removing things. You will have
to do some testing on your specific use case to check whether it will
work sufficiently well for you.
If it turns out to not be good enough, you may be better off using a
proper clustered file system (GFS/GFS2, OCFS/OCFS2 or GlusterFS). This
will provide consistent, non-racing results, but will come with a
performance penalty.
> 2. I'm using a Ubuntu 10.2 dist. Can I launch lsynd with any user, not
> with root? Because when I try to launch with other users the deamon
> doesn't start (only with root).
I think you need root privileges to register inotify watches.
Gordan
Gordan Bobic
> Hello David!
>
> > Are there any way to sync the files, using lsyng
> > from the source and the target in both directions? Can it cause any
> > problems such a infinite loop?
>
> Yes it can cause problems and cause infinite loops. Generally its not
> adviced to do this, you are going very much on the experimental side
> of things with this. However, practically it is possible, as e.g.
> Gordon here runs it that way. You will have to set rsync to use a tmp
> directory and not create files in place. Still there a unwanted race
> conditions if a directory changes at both sites at the same time.
The race conditions aren't limited to use-cases with writes hitting both
sites. It is possible to end up with unwanted file deletions or unwanted
non-deletions even if all writes/deletes are happening on one node. Here
are a couple of examples:
Example 1:
On node1, add 3 files, file1, file2, file3 very quickly (e.g. copy them)
This triggers rsync on that directory. file1 gets copied to node2. node2
notices a new file, and initiates rsync to node1. node2 only has file1
at this point.
node1's rsync is now copying file2.
node2 is rsyncing it's directory contents to node1. Since node2 only has
file1, file2 and file3 will be deleted on node1.
node1 is already copying file2, so this will get there since file handle
will remain active.
node2 gets file2, and triggers rsync. It copies file2 to node1.
node1 notices file2 has arrived, and rsyncs. This finds no differences
between the two nodes, so the situation stabilizes.
Net result: file3 has gotten silently deleted.
Example 2:
Take a big directory tree (e.g. kernel source). rm -rf it on node1. This
will trigger a rsync as soon as the first file is deleted. rsync will
start syncing to the other node (node2), which will in turn trigger
rsync back.
The problem is that on node2 still has all of the files, and will start
rsyncing files back to node1 even if those files are already deleted on
node1.
Net result: Some files that were "rm -rf"-ed will actually re-appear on
the node you initiated their removal. You may need to do it several
times before the situation stabilizes the way you want it to.
Note that this can also cause large bandwidth usage if the files in
question are big, and that rsync causes a LOT of CPU load, so this
bouncing will cause quite a lot of unproductive load. For example, a
1.6GHz Atom seems to top out at about 30MB/s with rsync using up 100% of
CPU. I'm looking into rebuilding rsync using ICC to try to get a bit
more performance out of it, but I am not sure how much gain it will
achieve without any source code changes.
> > 2. I'm using a Ubuntu 10.2 dist. Can I launch lsynd with any user, not
> > with root? Because when I try to launch with other users the deamon
> > doesn't start (only with root).
>
> Yes, you can do that, and I run lsyncd that way myself. You just have
> to have a user which has read access to all the files you watch.
Ah, that is good to know, thanks. :)
Gordan
David Fuentes Ruiz
Ok, I understand the problem with the sync in the two directions.
The second problem that i mentioned it was not exactly that i wrotte. It's the openssh who is requesting to the client the password.
I have generated a public rsa kay for the client and it's copied to the server. In the shell, password is not requested (I can do a ssh connection to the server without type any pass, and launch a rsync manually and the password it's not requested), but when I launch lsyncd and the rsync is launched, the openssh ask for the password at any event detected over the folder that it's being watched.
The configuration of my lsync file is the next:
<callopts>
<option text="-avz"/>
<option text="-e ssh"/>
<option text="--delete"/>
<exclude-file/>
<source/>
<destination/>
</callopts>
</settings>
<directory>
<source path="/home/administrator/dropfiles/"/>
<target path="192.168.1.204:/home/administrator/dropfiles/"/>
<!--
or it can also be an absolute path for localhost
<target path="/absolute/path/to/target">
-->
</directory>
In the connection, if i don't specify the user at the target line, it takes root as the default user not the user that is logged and running lsyncd.
Are there some mistake in the conf???
--
You received this message because you are subscribed to the Google Groups "lsyncd" group.
To post to this group, send email to lsy...@googlegroups.com.
To unsubscribe from this group, send email to lsyncd+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/lsyncd?hl=en.
Axel Kittenberger
trouble some. -l and -t are missing as well, but that might be your
decission.
Regarding "-e ssh" using spaces within options can be troublesome, it
is the same as using "-e ssh" in bash with quotes, forcing them
together.
<option text="-e"/> <option text="ssh"/> is safer (maybe rsync
recognizes the space I dont know.
Are you running lsyncd as the same user you configured the rsa keys with?
- Axel
David Fuentes Ruiz
I have tried with the default configuration of lsynd:
<callopts>
<option text="-lt%r"/>
<!-- <option text="-e"/>
<option text="ssh"/> -->
<option text="--delete"/>
<exclude-file/>
<source/>
<destination/>
</callopts>
If I specify a user with something like that:
<directory>
<source path="/home/administrator/dropfiles/"/>
<!--
or it can also be an absolute path for localhost
<target path="/absolute/path/to/target">
-->
</directory>
Axel Kittenberger
really runs with? I don't know in what way rsync determines the
default user, but well, the environment lsyncd provides when it calls
rsync is very minimal nothing to compare with bash. Maybe its the
"%USER" environment variable or something like that. Also look what
happens if you run lsyncd in --no-daemon mode.
- Axel
David Fuentes Ruiz
But the problem is that my app needs to run the lsyncd deamon when the applications it's installing (the last step). Because I want that if the user log out and them log in, the lsync deamon has to be running and with the no daemon option the process is killed by the system.
I has been googleing and I think the problem it's related with the ssh-agent needed to make the ssh authentication in Gnome process (graphic mode, not bash mode). I will try to find the solution for this, but seems that the problem not is the lsync daemon.
Axel Kittenberger
nohup /home/axel/lsyncd --no-daemon /home/axel/lsyncd-svn/src
/home/axel/lsyncd-svn/dst >/tmp/lsyncdlog 2>/tmp/lsyncdlog &
This way lsyncd will not daemonize itself, but will keep running when
you log out.
The idea of deamonize is exactly to strip the daemon of all the
callers user specific environmental stuff to get into a clean state,
in your case it seems to clean itself from something you need.
Otherwise I'd find another way for rsync to specify command lines to
find your passkeys.
- Axel