[Samba] winbind pam trouble

lists

unread,

Apr 11, 2016, 3:00:03 PM4/11/16

to

Hi,

I just upgraded my member (fileserver) server (wheezy) from sernet-4.1
to sernet-4.2, to be ready for tomorrow's badlock outbreak.

Under 4.1 we used sssd, and now 4.2 with winbind. Everything seems to be
running good: wbinfo (-p, -u, -g, -t) all give the expected results,
same for getent (group, passwd, username)

File serving works, life is good. :-)

Last step: allowing ssh access for AD users with a configured shell into
my member server -> PAM

I followed the list instructions: created the file
/usr/share/pam-configs/winbind with the content taken from the list.

Then run pam-auth-update, disabled SSS, enabled winbind. But alas...
logging in over ssh does not work, and auth.log tells me:

> Apr 11 20:18:32 filehost sshd[4884]: pam_winbind(sshd:auth): getting password (0x00000388)
> Apr 11 20:18:32 filehost sshd[4884]: pam_winbind(sshd:auth): pam_get_item returned a password
> Apr 11 20:18:32 filehost sshd[4884]: pam_winbind(sshd:auth): request wbcLogonUser failed: WBC_ERR_AUTH_ERROR, PAM error: PAM_SYSTEM_ERR (4), NTSTATUS: NT_STATUS_CONNECTION_DISCONNECTED, Error message was: The transport connection is now disconnected.
> Apr 11 20:18:32 filehost sshd[4884]: pam_winbind(sshd:auth): internal module error (retval = PAM_SYSTEM_ERR(4), user = 'username')
> Apr 11 20:18:34 filehost sshd[4884]: Failed password for username from x.y.z.88 port 49302 ssh2

Internal module error? WBC_ERR_AUTH_ERROR, PAM error: PAM_SYSTEM_ERR (4)?

Does anyone have an idea what is going on here?

MJ

--
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba

lists

unread,

Apr 11, 2016, 3:10:04 PM4/11/16

to

Seems I cheered too early, and I have some more winbind issues I didn't
realise before... here are winbind logs:

> [2016/04/11 20:39:01.330107, 1] ../source3/librpc/crypto/gse_krb5.c:416(fill_mem_keytab_from_system_keytab)
> ../source3/librpc/crypto/gse_krb5.c:416: krb5_kt_start_seq_get failed (Permission denied)
> [2016/04/11 20:39:01.330143, 0] ../lib/util/fault.c:78(fault_report)
> ===============================================================
> [2016/04/11 20:39:01.330173, 0] ../lib/util/fault.c:79(fault_report)
> INTERNAL ERROR: Signal 11 in pid 4899 (4.2.9-SerNet-Debian-8.wheezy)
> Please read the Trouble-Shooting section of the Samba HOWTO
> [2016/04/11 20:39:01.330199, 0] ../lib/util/fault.c:81(fault_report)
> ===============================================================
> [2016/04/11 20:39:01.330217, 0] ../source3/lib/util.c:788(smb_panic_s3)
> PANIC (pid 4899): internal error
> [2016/04/11 20:39:01.330733, 0] ../source3/lib/util.c:899(log_stack_trace)
> BACKTRACE: 29 stack frames:
> #0 /usr/lib/x86_64-linux-gnu/samba/libsmbconf.so.0(log_stack_trace+0x1a) [0x7f64c5f6699b]
> #1 /usr/lib/x86_64-linux-gnu/samba/libsmbconf.so.0(smb_panic_s3+0x55) [0x7f64c5f66a99]
> #2 /usr/lib/x86_64-linux-gnu/samba/libsamba-util.so.0(smb_panic+0x2d) [0x7f64c9883ed3]
> #3 /usr/lib/x86_64-linux-gnu/samba/libsamba-util.so.0(+0x231ec) [0x7f64c98841ec]
> #4 /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0) [0x7f64cb2520a0]
> #5 /usr/lib/x86_64-linux-gnu/samba/libkrb5-samba4.so.26(krb5_storage_free+0x4) [0x7f64c7f0ae4f]
> #6 /usr/lib/x86_64-linux-gnu/samba/libkrb5-samba4.so.26(+0x3c7cd) [0x7f64c7ef67cd]
> #7 /usr/lib/x86_64-linux-gnu/samba/libkrb5-samba4.so.26(krb5_kt_end_seq_get+0x2a) [0x7f64c7ef5eac]
> #8 /usr/lib/x86_64-linux-gnu/samba/libgse-samba4.so(+0xa981) [0x7f64c4aaf981]
> #9 /usr/lib/x86_64-linux-gnu/samba/libgse-samba4.so(gse_krb5_get_server_keytab+0x3db) [0x7f64c4aafdaa]
> #10 /usr/lib/x86_64-linux-gnu/samba/libgse-samba4.so(+0xc644) [0x7f64c4ab1644]
> #11 /usr/lib/x86_64-linux-gnu/samba/libgensec.so.0(gensec_start_mech+0x197) [0x7f64c4ce3eaf]
> #12 /usr/lib/x86_64-linux-gnu/samba/libgensec.so.0(gensec_start_mech_by_oid+0xd9) [0x7f64c4ce4194]
> #13 /usr/sbin/winbindd(kerberos_return_pac+0x5b2) [0x7f64cb6a8248]
> #14 /usr/sbin/winbindd(winbindd_dual_pam_auth+0x792) [0x7f64cb6c6be5]
> #15 /usr/sbin/winbindd(+0x5aa44) [0x7f64cb6dba44]
> #16 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0x9771) [0x7f64c9001771]
> #17 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0x7a2b) [0x7f64c8fffa2b]
> #18 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_once+0x92) [0x7f64c8ffc3b1]
> #19 /usr/sbin/winbindd(+0x5daef) [0x7f64cb6deaef]
> #20 /usr/sbin/winbindd(+0x5dc57) [0x7f64cb6dec57]
> #21 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0x4d68) [0x7f64c8ffcd68]
> #22 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(tevent_common_loop_immediate+0x128) [0x7f64c8ffcc15]
> #23 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0x94ba) [0x7f64c90014ba]
> #24 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0x7a2b) [0x7f64c8fffa2b]
> #25 /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_once+0x92) [0x7f64c8ffc3b1]
> #26 /usr/sbin/winbindd(main+0x11d5) [0x7f64cb6b7319]
> #27 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f64c3879ead]
> #28 /usr/sbin/winbindd(+0x26a09) [0x7f64cb6a7a09]
> [2016/04/11 20:39:01.330997, 0] ../source3/lib/dumpcore.c:313(dump_core)
> unable to change to /var/log/samba/cores/winbindd
> refusing to dump core

These errors sound serious and scary... A good idea, anyone?

lists

unread,

Apr 12, 2016, 3:10:03 AM4/12/16

to

Some other observations in log.winbindd-idmap:

> [2016/04/12 08:37:54.028456, 1] ../source3/winbindd/idmap_ad.c:523(idmap_ad_sids_to_unixids)
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-133237
> [2016/04/12 08:45:57.051863, 1] ../source3/winbindd/idmap_ad.c:523(idmap_ad_sids_to_unixids)
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-133222

This happens for 30 different SID's: some with a long last RID:

> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-133237
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-132270
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-132722

and with shorter RID's like
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-501
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-502
> Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-517

However, and looking at an ldif dump of our CN=Users, I can't find these
numbers...?

Anyone..?

L.P.H. van Belle

unread,

Apr 12, 2016, 3:40:03 AM4/12/16

to

I just looked over your previous messages.

I think the best if that you setup sssd again, so keep the setup as it was.

I just upgraded my sernet samba 4.2.7 to latest 4.2.9
And from that point i upped to 4.3.6 ( debian samba, a rebuild from debian sid to jessie )

This was without problems, but im not using sssd.

Maybe someone with sssd knowledge can help more why you have sid differendes.

Greetz,

Louis

> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-...@lists.samba.org] Namens lists
> Verzonden: dinsdag 12 april 2016 9:04
> Aan: sa...@lists.samba.org
> Onderwerp: Re: [Samba] winbind pam trouble

>
> Some other observations in log.winbindd-idmap:
>
> > [2016/04/12 08:37:54.028456, 1]
> ../source3/winbindd/idmap_ad.c:523(idmap_ad_sids_to_unixids)
> > Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-
> 133237
> > [2016/04/12 08:45:57.051863, 1]
> ../source3/winbindd/idmap_ad.c:523(idmap_ad_sids_to_unixids)
> > Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-
> 133222
>
> This happens for 30 different SID's: some with a long last RID:
>
> > Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-
> 133237
> > Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-

> 132270

> > Could not get unix ID for SID S-1-5-21-90839350-987482234-868425949-

Rowland penny

unread,

Apr 12, 2016, 4:00:03 AM4/12/16

to

You have real trouble if you don't have the last three :-D

They are well known SIDs

501 is Guest
502 is krbtgt
517 is Cert Publishers

Try opening a terminal on the DC and run this:

ldbsearch -H /usr/local/samba/private/sam.ldb
'(objectsid=S-1-5-21-90839350-987482234-868425949-501)'

This should display the AD object for the SID, provided you have
compiled Samba yourself or have installed ldb-tools and changed
'/usr/local/samba/private' for the path to your sam.ldb.

Repeat for the other SIDs.

Rowland

lists

unread,

Apr 12, 2016, 4:30:04 AM4/12/16

to

Hi Rowland,

> You have real trouble if you don't have the last three :-D
>
> They are well known SIDs
>
> 501 is Guest
> 502 is krbtgt
> 517 is Cert Publishers
>
> Try opening a terminal on the DC and run this:
>
> ldbsearch -H /usr/local/samba/private/sam.ldb
> '(objectsid=S-1-5-21-90839350-987482234-868425949-501)'

I searched with ldbsearch, and that confirmed that we DO have those
records. So at least we don't seem to have REAL trouble. :-D

Looking at ADUC, I realise that these 'problem' acounts are the (few)
accounts with no UID assigned to them. So the 'error' makes sense: they
are mailinglists, or groups not used for file access permissions.

So it seems this is logical, and does not explain the problems we had
yesterday evening with winbind crashing, as I wrote in my second email
yesterday:

Any ideas where to look next..?

One line that struck me in the loglines above is:

> INTERNAL ERROR: Signal 11 in pid 4899 (4.2.9-SerNet-Debian-8.wheezy)

Debian 8, wheezy.... strange to see those two (8, wheezy) in one line.

We're on wheezy, and my sources.list line is also for wheezy.

Suggestions?

MJ

lists

unread,

Apr 12, 2016, 4:40:04 AM4/12/16

to

Hi Louis,

On 12-4-2016 9:31, L.P.H. van Belle wrote:
> I just looked over your previous messages.
>
> I think the best if that you setup sssd again, so keep the setup as it was.

I did that for now. However, I would like to (eventually) use winbind
rather than sssd, to become as normal as possible. :-D

Rowland penny

unread,

Apr 12, 2016, 4:40:04 AM4/12/16

to

apt-get install libpam-krb5

Rowland

If I login to a domain member via ssh I get this in /var/log/auth.log:

Apr 12 09:21:21 member1 sshd[6502]: pam_krb5(sshd:auth): user rowland
authenticated as row...@SAMDOM.EXAMPLE.COM
Apr 12 09:21:22 member1 sshd[6500]: Accepted keyboard-interactive/pam
for rowland from 192.168.0.128 port 41609 ssh2
Apr 12 09:21:22 member1 sshd[6500]: pam_unix(sshd:session): session
opened for user rowland by (uid=0)

Rowland

L.P.H. van Belle

unread,

Apr 12, 2016, 5:10:03 AM4/12/16

to

Ok,

Rowland i just checked the wheezy packages, these are ok.
My DC's are still running wheezy all other are Jessie now.

Check What is installed?
dpkg -l | egrep "krb5-user|libnss|libpam|winbind|samba|sernet " > sambapackages.txt

Backup /etc/samba
Backup /var/lib/samba

apt-get remove sernet-*

dpkg -l | grep sernet
remove the last things

Install samba again and let the installer fix the dependecies.

Check for needed packages again.
dpkg -l | egrep "krb5-user|libnss|libpam|winbind|samba|sernet"
missing something install manualy.

Reboot.

If you problem existe because of faulty pakcages that should be fixed with above.
( you can use the reinstall options, but i prefferer to remove )

Greetz,

Louis

> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-...@lists.samba.org] Namens lists

> Verzonden: dinsdag 12 april 2016 10:28

> Aan: sa...@lists.samba.org
> Onderwerp: Re: [Samba] winbind pam trouble
>

lists

unread,

Apr 12, 2016, 8:00:04 AM4/12/16

to

On 12-4-2016 10:24, lists wrote:

From log.wb-DOMAIN:

>
> [2016/04/11 20:39:01.330173, 0] ../lib/util/fault.c:79(fault_report)
> INTERNAL ERROR: Signal 11 in pid 4899 (4.2.9-SerNet-Debian-8.wheezy)
> Please read the Trouble-Shooting section of the Samba HOWTO
> [2016/04/11 20:39:01.330199, 0] ../lib/util/fault.c:81(fault_report)
> ===============================================================
> [2016/04/11 20:39:01.330217, 0] ../source3/lib/util.c:788(smb_panic_s3)
> PANIC (pid 4899): internal error
> [2016/04/11 20:39:01.330733, 0]
> ../source3/lib/util.c:899(log_stack_trace)
> BACKTRACE: 29 stack frames:

I am trying to trigger the above panic:

wbinfo -u (lists all users)
wbinfo -g (lists all groups)
wbinfo -t (checking trust succeeds)
wbinfo --list-status (all three reported online)
wbinfo --authenticate=user%password (authentication succeeds)
wbinfo --dc-info=DOMAIN (lists only ONE of our three DCs, but no panic)

The above panic happened yesterday while nsswitch.conf contained
> passwd: compat winbind
> group: compat winbind
> shadow: compat winbind

Currently nsswitch.conf has
> passwd: compat sss
> group: compat sss
> shadow: compat sss
and winbind is running without crashes.

Since this is production, I'd rather not put back winbind in
nsswitch.conf, because the winbind panic will break fileserving from
this member server.

Any ideas how to approach this..?

BTW: wheezy 7.10, x64, sernet-samba 4.2.9-8.

Rowland penny

unread,

Apr 12, 2016, 8:20:03 AM4/12/16

to

I never alter 'shadow' and winbind doesn't crash for me i.e. relevant
part of /etc/nsswitch.conf:

passwd: compat winbind
group: compat winbind
shadow: compat

wheezy 7.10, x64, sernet-samba 4.2.4-7

I will upgrade sernet samba and see what happens.

Rowland

Rowland penny

unread,

Apr 12, 2016, 8:40:05 AM4/12/16

to

OK, now using sernet-samba 4.2.9 and no difference, everything still
works, I can login via ssh to another machine and I can login via ssh
from another machine.

lists

unread,

Apr 12, 2016, 9:30:04 AM4/12/16

to

Hi Rowland,

On 12-4-2016 14:28, Rowland penny wrote:
>> I never alter 'shadow' and winbind doesn't crash for me i.e. relevant
>> part of /etc/nsswitch.conf:
>>
>> passwd: compat winbind
>> group: compat winbind
>> shadow: compat
>>
>> wheezy 7.10, x64, sernet-samba 4.2.4-7
>>
>> I will upgrade sernet samba and see what happens.
>>
>> Rowland
>>
>>
>
> OK, now using sernet-samba 4.2.9 and no difference, everything still
> works, I can login via ssh to another machine and I can login via ssh
> from another machine.

Thanks for testing. I'll try without winbind for shadow, hopefully tonight.

MJ

mj

unread,

Apr 14, 2016, 3:10:04 AM4/14/16

to

Hi Rowland, list,

On 04/12/2016 02:28 PM, Rowland penny wrote:
>> I never alter 'shadow' and winbind doesn't crash for me i.e. relevant
>> part of /etc/nsswitch.conf:
>>
>> passwd: compat winbind
>> group: compat winbind
>> shadow: compat
>>
>> wheezy 7.10, x64, sernet-samba 4.2.4-7
>>
>> I will upgrade sernet samba and see what happens.

So, I took out the shadow, tried winbind again, and so far so good: no
panics yet, and also ssh works immediately! :-)

For now I guess winbind has serious difficulties handling shadow requests...

Thanks for the feedback!

MJ