G'day Sven,
On 27/06/2013, at 5:05 PM, Sven Breuner wrote:
> Hi David,
>
> we haven't tried Dovecot yet and I'm not sure whether the two problems (too many open files and symlinks) are related.
>
> So regarding the number of open files: Are you runnig the fhgfs server daemons as user or with root privileges? I don't know how many files Dovecot usually tries to open simultaneously, but user processes are normally limited to about 1024 open files (and the fhgfs servers are stateful, so they also need to keep the chunkfile file open for every file that is open on a client). If the fhgfs servers have root privileges, they automatically try to increase their limit to 50000.
> The currently effective limit can be checked with "ulimit -a" and can be changed via sysctl.
> (Of course, there is also a chance that this is a bug in dovecut where it forgets to close files in situations where something else is going wrong.)
>
The FhGFS server services are running as root.
[root@kwd-parallelfs-01 ~]# ulimit -a
open files (-n) 1024
After changing the "sysCreateHardlinksAsSymlinks" option as suggested I'm not getting the "Too many open files" message. However....
> Regarding the dangling symlinks: Do you know if Dovecut tries to create hardlinks? Because they are not supported by fhgfs yet (which is subject to change with the upcoming releases).
> As a workaround, the fhgfs client by default creates symlinks when an application tries to create a hardlink - which works well for some apps, but not so well for others.
> Some applications also have their own workarounds built in when they see that a file system does not support hardlinks, so you might want to try changing the corresponding option in /etc/fhgfs/fhgfs-client.conf tosysCreateHardlinksAsSymlinks=false, which will result in an appropriate error code instead of a symlink when an application tries to create a hardlink.
>
Dovecot does by default use hardlinks for file copies. I have set it to not do this.
/etc/dovecot/conf.d/10-mail.conf:
maildir_copy_with_hardlinks = no
Now we're getting sets of messages like this:
Jun 27 17:57:20 kwd-parallelfs-client-01 dovecot: imap(david): Error: link(/kusers/techies/david/Maildir/Junk_To_Sort/Other/.temp.kwd-parallelfs-client-01.scem.uws.edu.au.1918.bc5446b8e71c817c, /kusers/techies/david/Maildir/Junk_To_Sort/Other/dovecot-uidlist.lock) failed: Operation not permitted
Jun 27 17:57:20 kwd-parallelfs-client-01 dovecot: imap(david): Error: file_dotlock_create(/kusers/techies/david/Maildir/Junk_To_Sort/Other/dovecot-uidlist) failed: Operation not permitted
It looks like Dovecot is trying to use hardlinks for other things......
Any e.t.a. on the hardlink update?
Cheers,
> --
> You received this message because you are subscribed to the Google Groups "fhgfs-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
fhgfs-user+...@googlegroups.com.
> For more options, visit
https://groups.google.com/groups/opt_out.
>
>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
David Minard.
Ph: 0247 360 155
Fax: 0247 360 770
School of Computing, Engineering, and Mathematics
Building Y - Penrith Campus (Kingswood)
Locked bag 1797
Penrith South DC
NSW 1797
[Sometimes waking up just isn't worth the insult of the day to come.]
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.