AW: winbind 3.3.7 memory usage
Dr. Hansjoerg Maurer
If run winbind with valgrind for 4 days no
One winbind (PID 14762) grows up to 727/448 MByte VSZ/RSS
[root@donau ~]# psgrep winbind
root 11749 0.1 5.7 394492 119072 ? Ss Oct01 10:09 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 11806 0.0 3.0 351088 62516 ? S Oct01 0:05 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 11807 0.0 3.2 357580 67176 ? S Oct01 0:06 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 12053 0.0 3.3 355820 68288 ? S Oct01 0:14 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 12054 0.0 4.5 370644 92540 ? S Oct01 0:39 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 14502 0.2 5.0 393100 104816 ? S Oct01 15:04 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
root 14762 6.6 21.8 727332 448924 ? R Oct01 371:14 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
The valgrind output has a uncompressed size of 440 MByte's (compressed 4 MByte)
therefore I put it to
ftp://ftp.dlr.de/robotic/outgoing/maurerh/valgrind.out.bz2
Regards
Hansjörg
----- Originalnachricht -----
Von: Volker Lendecke <Volker....@SerNet.DE>
Gesendet: Don, 1.10.2009 10:29
An: Dr. Hansjoerg Maurer <hansjoer...@itsd.de>
Cc: samba-t...@lists.samba.org
Betreff: Re: winbind 3.3.7 memory usage
On Thu, Oct 01, 2009 at 10:18:54AM +0200, Dr. Hansjoerg Maurer wrote:
> yes, no problem, we have a box where this should be no problem.
> Can you point me to a short howto, how I can do this?
Install debugging symbols. Either your distro should bring
samba debuginfo packages, or if you compile it yourself,
compile with -g. ./configure --enable-developer for example
does that.
Install valgrind (www.valgrind.org or your distro repo).
Then start winbind as follows (all in one line of course):
valgrind --tool=memcheck --leak-check=full -v
--num-callers=20 --trace-children=yes winbindd >
/tmp/valgrind.out 2>&1
The resulting processes will be larger from the start. If
after a while one subprocess has grown substantially, shut
down winbind with
smbcontrol <pid> stop
The output of /tmp/valgrind.out is the interesting piece
then.
Volker
Volker Lendecke
> If run winbind with valgrind for 4 days no
> One winbind (PID 14762) grows up to 727/448 MByte VSZ/RSS
>
> [root@donau ~]# psgrep winbind
> root 11749 0.1 5.7 394492 119072 ? Ss Oct01 10:09 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 11806 0.0 3.0 351088 62516 ? S Oct01 0:05 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 11807 0.0 3.2 357580 67176 ? S Oct01 0:06 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 12053 0.0 3.3 355820 68288 ? S Oct01 0:14 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 12054 0.0 4.5 370644 92540 ? S Oct01 0:39 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 14502 0.2 5.0 393100 104816 ? S Oct01 15:04 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
> root 14762 6.6 21.8 727332 448924 ? R Oct01 371:14 valgrind --tool=memcheck --leak-check=full -v --num-callers=20 --trace-children=yes winbindd
>
>
> The valgrind output has a uncompressed size of 440 MByte's (compressed 4 MByte)
Ok, that one seems like a valgrind artifact. It seems not to
complain about something in the Kerberos libraries, which
are, as we all know, perfect. (sorry...).
The only significant memory leak as reported by valgrind is
in the idmap child:
==14502== 83,415 bytes in 4,020 blocks are definitely lost in loss record 61 of 67
==14502== at 0x4904A06: malloc (vg_replace_malloc.c:149)
==14502== by 0x3B8E570FF1: strdup (in /lib64/tls/libc-2.3.4.so)
==14502== by 0x14A71A61: ???
==14502== by 0x14A7EBA4: ???
==14502== by 0x14A85CAC: ???
==14502== by 0x14A864F0: ???
==14502== by 0x14A81CAD: ???
==14502== by 0x147C51A7: ???
==14502== by 0x147C7294: ???
==14502== by 0x3B8E58E75E: getgrnam_r@@GLIBC_2.2.5 (in /lib64/tls/libc-2.3.4.so)
==14502== by 0x3B8E58DDBE: getgrnam (in /lib64/tls/libc-2.3.4.so)
==14502== by 0x391938: ??? (idmap_nss.c:189)
==14502== by 0x387E72: idmap_backends_sid_to_unixid (idmap.c:785)
==14502== by 0x388F0C: idmap_sid_to_gid (idmap_util.c:259)
==14502== by 0xBFBE7: winbindd_dual_sid2gid (winbindd_idmap.c:374)
==14502== by 0xB71B8: ??? (winbindd_dual.c:453)
==14502== by 0xB7971: ??? (winbindd_dual.c:214)
==14502== by 0xB7E2E: ??? (winbindd_dual.c:189)
==14502== by 0x1445CB: run_events (events.c:233)
==14502== by 0x85961: main (winbindd.c:850)
It seems that on your system getgrnam internally leaks
memory. There's nothing Samba can do about this.
Hope that helps,
Volker
Danilo Almeida
<quote from="Volker">
</quote>
This might be a nsswitch module. You might try looking at what modules you have in /etc/nsswitch.conf for groups and see if that might point you in the right direction. If you have a non-default module, that might be the first one to investigate.
- Danilo
Dr. Hansjoerg Maurer
bingo, very good point.
The unix systems are integrated into AD using Vintella Authentication Services from Quest.
Therefore they use the vas nss module in /etc/nsswitch.conf
Sorry, I did not include this information in the initial mail, but I did not realize,
that this might be relevant.
I will open a call by quest and inform the list, when they find a solution in order to document
the problem.
Thank you very much
Regards
hansjörg
----- Originalnachricht -----
Von: Danilo Almeida <dalm...@likewise.com>
Gesendet: Mit, 7.10.2009 07:32
An: Dr. Hansjoerg Maurer <hansjoer...@itsd.de> ; Volker....@SerNet.DE
Cc: samba-t...@lists.samba.org
Betreff: RE: winbind 3.3.7 memory usage
Dave Daugherty
May I ask what OS platform and version you are running Samba on? I see glibc_2.2.5 in the stack trace.
It seems a little odd that getgrnam_r would be allocating memory, since it is given a buffer to stuff things into.
We also use this function and I would like to test ourselves.
Regards,
Dave Daugherty
Centrify Corp.
~~~~~~~~~~~~~
Sent: Tuesday, October 06, 2009 11:55 PM
To: Danilo Almeida; Volker....@SerNet.DE
Cc: samba-t...@lists.samba.org
Subject: AW: winbind 3.3.7 memory usage
Dr. Hansjoerg Maurer
the problem occurs on RHEL4 and RHEL5 (or CentOS), both x86_64.
On a 32 bit testmachine, it seems no problem,
which may also be related, that nobody is workin on it
Here the i386 Plattform
[root@rmvrhli3-ha ~]# uname -a
Linux rmvrhli3-ha.robotic.dlr.de 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:19:18 EDT 2009 i686 i686 i386 GNU/Linux
[root@rmvrhli3-ha ~]# psgrep winbind
root 25180 0.0 0.4 197472 4936 ? Ss Aug20 4:51 winbindd
root 25181 0.0 0.7 197248 7196 ? S Aug20 0:49 winbindd
root 25184 0.0 0.5 19788 5172 ? S Aug20 1:13 winbindd
root 25282 0.0 0.1 15612 1388 ? S Aug20 0:03 winbindd
root 27862 0.0 0.2 16128 2060 ? S 08:33 0:00 winbindd
root 27863 0.0 0.1 16128 1904 ? S 08:33 0:00 winbindd
root 27864 0.0 0.4 16360 4668 ? S 08:33 0:00 winbindd
root 27961 1.0 0.4 197604 4136 ? S 08:39 0:00 winbindd
Here the x86_64 Plattform
[root@rmcs31 ~]# uname -a
Linux rmcs31 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
[root@rmcs31 ~]# psgrep winbind
root 10654 0.0 0.2 303984 9204 ? Ss Aug26 2:56 winbindd
root 10655 0.0 0.4 356824 20120 ? S Aug26 1:57 winbindd
root 10669 0.0 27.3 5387512 1104628 ? S Aug26 29:39 winbindd
root 10670 0.0 0.0 302884 748 ? S Aug26 0:01 winbindd
root 10751 0.0 0.0 302896 364 ? S Aug26 0:00 winbindd
root 10769 0.0 0.0 304888 636 ? S Aug26 0:00 winbindd
Regards
Hansjörg
----- Originalnachricht -----
Von: Dave Daugherty <dave.da...@centrify.com>
Gesendet: Mit, 7.10.2009 19:41
An: Dr. Hansjoerg Maurer <hansjoer...@itsd.de> ; Danilo Almeida <dalm...@likewise.com> ; Volker....@SerNet.DE