How many files and directories does your lsyncd handle well?

1,047 views
Skip to first unread message

Sven Hartrumpf

unread,
Dec 28, 2010, 5:08:33 AM12/28/10
to lsy...@googlegroups.com
Hi all.

I have got the impression that inotify can be seen as a bottleneck of lsyncd.
(Please correct me if I am wrong.)

So, what is your experience?
How many files and directories can you handle with lsyncd?

For example:
Is 1 TB in 5-10 million files with 1%-10% change per month (over Gb Ethernet) ok?

Greetings
Sven

Axel Kittenberger

unread,
Dec 28, 2010, 6:26:34 AM12/28/10
to lsy...@googlegroups.com
Hi!

Lsyncd adds one inotify watch per directory, so its the amount of
directories matter than the amount of files. The Linux kernel uses 540
bytes or 1kB on 64bit unswapable kernel memory for every watch. Thats
the main bottle neck in matter of size of the watched tree. So
multiply that with the amount of directories. I cannot say exactly how
much user memory each watch takes in Lsyncd itself, it depends on the
path sizes.

Otherwise its hard to say a magic number, it depends on your configuration.

Kind regards, Axel

> --
> You received this message because you are subscribed to the Google Groups
> "lsyncd" group.
> To post to this group, send email to lsy...@googlegroups.com.
> To unsubscribe from this group, send email to
> lsyncd+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/lsyncd?hl=en.
>

Sven Hartrumpf

unread,
Jan 24, 2011, 4:09:12 AM1/24/11
to lsy...@googlegroups.com
Hi all.

I asked some weeks ago:


> How many files and directories can you handle with lsyncd?

> Is 1 TB in 5-10 million files with 1%-10% change per month (over Gb Ethernet)
> ok?

I am running now such a setup (but without Ethernet involved, since SOURCE and
TARGET are on the same computer, currently) with lsyncd 2.0.2 as packaged
for openSUSE 11.2.
lsyncd uses around 1000 inotify's, and it works perfectly!
I am impressed. Thanks for this tool and the help, Axel.

How can we contribute documentation extensions/fixes for the manual in the wiki?

Greetings
Sven

Axel Kittenberger

unread,
Jan 24, 2011, 4:24:39 AM1/24/11
to lsy...@googlegroups.com
> lsyncd uses around 1000 inotify's, and it works perfectly!
> I am impressed. Thanks for this tool and the help, Axel.

Thank you.

> How can we contribute documentation extensions/fixes for the manual in the wiki?

I added you (hart...@gmail.com) to the editors list, you can click
edit in the wiki. Please feel brave to do whatever you think improves
the documentation.

Kind regards,
Axel

Reply all
Reply to author
Forward
0 new messages