puppet slow when ensuring user with groups in combination with winbind auth

[Jan van Lith]

Hi,

I am using winbind with "winbind enum groups = yes" on some of our servers.
When ensuring a user that is local (and also in AD, so it has a lot of groups) the puppet run takes ages. Winbind process is taking a lot of cpu and when I strace it AD groups are passing by.

This is the manifest:

 # ensure user and group 
    user { user:
      ensure     => 'present',
      groups     => 'logongroup',
      uid        => '900',
      require    => Group[$user],
      managehome => true,
    }
    group { user:
      ensure => 'present',
      gid    => '900',
    }

# id user
uid=900(user) gid=900(user) groups=900(user),400(logongroup),16777729(domain users) .............

What is puppet doing? I am presuming it is checking if this user is a member of the logongroup.

Can you make puppet not performing these group checks preforming in AD?
My nsswitch.conf tells it to first look in local files.

passwd:     files winbind
shadow:     files winbind
group:      files winbind

So why is it still performing these tasks when the logongroup is already present in local files?

[jcbollinger]

On Monday, August 11, 2014 8:50:36 AM UTC-5, Jan van Lith wrote:

Hi,

I am using winbind with "winbind enum groups = yes" on some of our servers.
When ensuring a user that is local (and also in AD, so it has a lot of groups) the puppet run takes ages. Winbind process is taking a lot of cpu and when I strace it AD groups are passing by.

This is the manifest:

 # ensure user and group 
    user { user:
      ensure     => 'present',
      groups     => 'logongroup',
      uid        => '900',
      require    => Group[$user],
      managehome => true,
    }
    group { user:
      ensure => 'present',
      gid    => '900',
    }

# id user
uid=900(user) gid=900(user) groups=900(user),400(logongroup),16777729(domain users) .............

What is puppet doing?

It is likely enumerating all the groups defined for the machine, which it will do at the beginning of a run as part of determining the machine's initial state.  If winbind allows groups to be enumerated (as you specifically say it does for these machines) then those will include all the groups winbind can enumerate from AD.  Since you're using the name service switch, Puppet probably can't even tell that it's getting both AD groups and local groups.

 

I am presuming it is checking if this user is a member of the logongroup.

Puppet likely uses the 'groups' command to load users' secondary groups.  It might be that that requires scanning all AD groups (it does require scanning all local groups).  If determining a user's secondary groups generally takes a long time in a given environment, then there's probably nothing you can do to make Puppet do the job faster than is generally required.

Moreover, Puppet probably determines the secondary groups for all system users, which means the cost of running 'groups' is likely multiplied by the number of defined system users.  Furthermore, the known system users include those who are not permitted to log on, so that could extend to all users in AD.

 

Can you make puppet not performing these group checks preforming in AD?

Sure, by disabling winbind in nsswitch.conf.  But you probably don't want to do that.  Likely disabling group enumeration by winbind would also speed things up, but (1) you probably have it enabled for a reason, (2) Puppet probably then will not be able to determine users' membership in AD secondary groups, and (3) AD secondary groups might not work at all, at least for local users.

 

My nsswitch.conf tells it to first look in local files.

The problem is likely tied to the fact that by using winbind for groups at all, you add a gazillion groups to your system.  Name resolution precedence doesn't change that.

 

passwd:     files winbind
shadow:     files winbind
group:      files winbind

So why is it still performing these tasks when the logongroup is already present in local files?

It's probably not specific to this user, and almost certainly not to the 'logongroup' group.


John

[Garrett Honeycutt]

Hi Jan,

I'm not familiar with winbind itself though your performance might
improve by using nscd to cache the lookups.

and there's a module for that :)

https://github.com/ghoneycutt/puppet-module-nscd

Best regards,
-g


--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[jcbollinger]

On Wednesday, August 13, 2014 10:34:30 PM UTC-5, Garrett Honeycutt wrote:

I'm not familiar with winbind itself though your performance might
improve by using nscd to cache the lookups.

Although in principle a name service cache might help with the performance issue, the winbind docs say "Do not under any circumstances run nscd on any system on which winbindd is running."  I don't know the reason for this prohibition, but it seems both clear and forceful.


John