Jira (PUP-1153) Incorrect ordering of group and user resource purging

14 views
Skip to first unread message

Stephen Gelman (JIRA)

unread,
Mar 17, 2015, 12:20:40 PM3/17/15
to puppe...@googlegroups.com
Stephen Gelman commented on Bug PUP-1153
 
Re: Incorrect ordering of group and user resource purging

We are running into this exact problem. It seems like the status is "Needs Information" but I'm not sure what that information is. If someone could point me in the right direction I'd be happy to try and find the required information to get the ball rolling on fixing this.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a)
Atlassian logo

Stephen Gelman (JIRA)

unread,
May 13, 2015, 5:09:25 PM5/13/15
to puppe...@googlegroups.com

Kylo Ginsberg what is the hold up on this getting fixed? It's not necessarily a breaking issue but it does consistently cause errors in our puppet runs...

This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)
Atlassian logo

Kylo Ginsberg (JIRA)

unread,
May 13, 2015, 6:45:21 PM5/13/15
to puppe...@googlegroups.com
Kylo Ginsberg commented on Bug PUP-1153

In terms of general approach to the issue, I agree with the comment above to the effect that puppet shouldn't try to override /etc/login.defs.

Stephen Gelman can you say more about setting USERGROUPS_ENAB to no:
1) does it do what the man pages says it does? E.g. if you run the commands standalone outside of puppet?
2) does a puppet snippet like those given above still cause the group to be deleted when the last user is deleted?

Also can you report what distro you're trying this on? (Maybe there's some distro variance at play here as well ...)

If (1) is a yes, and (2) is a no, that will be weird (since puppet wraps those user* commands) but it will be some guidance on this issue.

Stephen Gelman (JIRA)

unread,
May 13, 2015, 8:17:20 PM5/13/15
to puppe...@googlegroups.com

Kylo Ginsberg, sorry, I should have given more detail. The reason it doesn't work is that the problem in puppet is different from the problem that USERGROUPS_ENAB is intended to solve. The purpose of USERGROUPS_ENAB=no is that when a user is deleted their primary group is deleted also (if it had no other members). The problem in puppet is that it tries to delete the group before the user. This fails because groupdel does not allow a user's primary group to be deleted. From its man page:

You may not remove the primary group of any existing user. You must remove the user before you remove the group.

I believe the solution here is that puppet should always delete users before deleting groups. I don't think it necessarily makes a difference but we are running Debian wheezy.

Stephen Gelman (JIRA)

unread,
Jun 15, 2015, 11:39:53 AM6/15/15
to puppe...@googlegroups.com

Charlie Sharpsteen (JIRA)

unread,
Jun 15, 2015, 11:43:07 AM6/15/15
to puppe...@googlegroups.com

Stephen Gelman (JIRA)

unread,
Sep 17, 2015, 11:42:35 PM9/17/15
to puppe...@googlegroups.com
 
Re: Incorrect ordering of group and user resource purging

Kylo Ginsberg: this ticket has been open since 2013 and affects us daily. Any chance of it getting prioritized to be fixed? AFAIK it would affect anyone running debian or ubuntu.

This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4)
Atlassian logo

Kylo Ginsberg (JIRA)

unread,
Sep 18, 2015, 8:56:03 PM9/18/15
to puppe...@googlegroups.com
Kylo Ginsberg commented on Bug PUP-1153

Stephen Gelman: first, apologies for not responding to your ping earlier this summer. But, better late than never

Second, re priorities, we generally pull in tickets that have lots of votes (or are implied by other tickets). This has just one vote, which is why it hasn't climbed high in our backlog. There isn't a magic threshold on votes, btw, but it's definitely something we look at.

But, and this is speculation, I'm wondering if this has the same root cause as PUP-1963, which we are currently working on. Again, that's speculation, but they seem related in that resources are generated, and several of the iterations above are trying to set dependencies that apparently aren't being honored. So .... if you can repro this running from source, you could try the PR for that ticket, or you could try once we release the fix for that (probably a few weeks out).

Hope some of that helps!

Stephen Gelman (JIRA)

unread,
Oct 21, 2015, 11:21:03 PM10/21/15
to puppe...@googlegroups.com

Well it now has 2 votes. Looking at PUP-1963 my gut feeling is that it is not the issue however I am going to test out the PR. I'll report back my findings.

Stephen Gelman (JIRA)

unread,
Jun 24, 2016, 12:28:05 PM6/24/16
to puppe...@googlegroups.com

Kylo Ginsberg: We just upgraded to puppet 4.5.2 and the issue is now worse. Before, the group removal errored but the user removal succeeded so the issue went away on subsequent puppet runs. Now, the group removal failing causes cascading dependency failures and the user removal gets skipped. This means the only way we are able to remove users on the box is doing so by hand. Is it possible for this to be prioritized more now that it is a huge breaking issue for us, and presumably anyone recursively managing users and groups in debian/ubuntu?

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)
Atlassian logo

Franz Skale (JIRA)

unread,
Jul 15, 2016, 2:33:04 AM7/15/16
to puppe...@googlegroups.com
Franz Skale commented on Bug PUP-1153

Kylo Ginsberg: We own a enterprise license and ran into the same problem. This is so annyoing, because it's a fact that prior deleting the primary group, you've to delete the user first.
That BUG is stopping our prododuction env.. The next problem is, when the exitvalue of userdel != 0 the user will not be removed from the secondary groups he's in.
We use hiera and my setup is as follows:
Debian 8 Agents
PE 2016.2.0

E.g.
Group:
"testgroup" :

{ "ensure" : "present", "gid" : 10005, "require" : "User[testuser]" }

User:
"testuser" :

{ "shell" : "/bin/false", "comment" : "", "home" : "/home/testuser", "password" : "!", "groups": ["ftpadmins", "webadmins"], "uid" : "10005", "gid" : "10005", "password_max_age" : "99999", "password_min_age" : "0" }

Delete the user via ensure = absent.
What happens

Jul 15 08:14:47 69-host puppet-agent[17425]: Could not delete group testgroup: Execution of '/usr/sbin/groupdel testgroup' returned 8: groupdel: cannot remove the primary group of user 'testuser'

What is weird is, that the primary group has been removed leaving the secondary group membership debris alive.

So, what has to be done to get things working ?

Do i have to write my own class ?
Do i have to patch the commands in puppet ruby classes ?

Right now i have to manually remove the debris.

Franz Skale (JIRA)

unread,
Jul 15, 2016, 3:05:06 AM7/15/16
to puppe...@googlegroups.com
Franz Skale commented on Bug PUP-1153

When setting USERGROUPS_ENAB=no, the secondary groups get removed but the primary group will be left.
When creating a user and set the require option, the user will not be created because the user will be created before the group.
Jul 15 08:57:55 69-host puppet-agent[25767]: Could not create user testuser: Execution of '/usr/sbin/useradd -g 10004 -G ftpadmins -d /nonexistent -p ! -s /bin/false -u 10004 testuser' returned 6: useradd: group '10004' does not exist
Jul 15 08:57:55 69-host puppet-agent[25767]: (/Stage[main]/User/User[testuser]/ensure) change from absent to present failed: Could not create user testuser: Execution of '/usr/sbin/useradd -g 10004 -G ftpadmins -d /nonexistent -p ! -s /bin/false -u 10004 testuser' returned 6: useradd: group '10004' does not exist

How can i distiguish between add (ensure=present) and delete (ensure=absent) in a defined type ?

Franz Skale (JIRA)

unread,
Jul 15, 2016, 3:23:10 AM7/15/16
to puppe...@googlegroups.com
Franz Skale updated an issue
 
Change By: Franz Skale
Comment:
When setting USERGROUPS_ENAB=no, the secondary groups get removed but the primary group will be left.
When creating a user and set the require option, the user will not be created because the user will be created before the group.
Jul 15 08:57:55 69-host puppet-agent[25767]: Could not create user testuser: Execution of '/usr/sbin/useradd -g 10004 -G ftpadmins -d /nonexistent -p ! -s /bin/false -u 10004 testuser' returned 6: useradd: group '10004' does not exist
Jul 15 08:57:55 69-host puppet-agent[25767]: (/Stage[main]/User/User[testuser]/ensure) change from absent to present failed: Could not create user testuser: Execution of '/usr/sbin/useradd -g 10004 -G ftpadmins -d /nonexistent -p ! -s /bin/false -u 10004 testuser' returned 6: useradd: group '10004' does not exist

How can i distiguish between add (ensure=present) and delete (ensure=absent) in a defined type ?

Jamie Campbell (JIRA)

unread,
Aug 24, 2016, 1:35:03 PM8/24/16
to puppe...@googlegroups.com
Jamie Campbell commented on Bug PUP-1153
 
Re: Incorrect ordering of group and user resource purging

This is an annoying issue that in my opinion has been unresolved for too long. I am having to manually remove users from nodes. One way to accomplish that without having to log into each server is via a salt command. If I am going to start using salt to clean up what Puppet can't do properly, well... You see where I'm headed with that?

Moses Mendoza (JIRA)

unread,
Sep 22, 2016, 3:22:18 PM9/22/16
to puppe...@googlegroups.com
Moses Mendoza updated an issue
 
Change By: Moses Mendoza
Labels: redmine  user-group-management
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Moses Mendoza (JIRA)

unread,
Sep 22, 2016, 3:25:03 PM9/22/16
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Labels: redmine manage-  user-group -management  redmine

Franz Skale (JIRA)

unread,
Sep 23, 2016, 1:14:04 AM9/23/16
to puppe...@googlegroups.com
Franz Skale commented on Bug PUP-1153
 
Re: Incorrect ordering of group and user resource purging

Hi Jamie,
perhaps i can help you.
Setting "USERGROUPS_ENAB no" in file /etc/login.defs worked out for me.
But USERGROUPS_ENAB yes is default for all distributions.
So IMHO puppet has to take care about setting it to "no" when managing user/groups.
There are pros and cons about this particular setting.
If you want to manager user/groups locally you have to take care to issue a groupdel after userdel if using the setting "USERGROUPS_ENAB no".
Rgds.
Franz

John Duarte (JIRA)

unread,
May 15, 2017, 2:29:05 PM5/15/17
to puppe...@googlegroups.com
John Duarte updated an issue
 
Change By: John Duarte
Labels: manage-user-group redmine  triaged

Moses Mendoza (JIRA)

unread,
May 18, 2017, 1:45:16 PM5/18/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Labels: manage-user-group redmine  triaged

Geoff Nichols (JIRA)

unread,
Mar 24, 2018, 7:40:04 PM3/24/18
to puppe...@googlegroups.com
Geoff Nichols updated an issue
Change By: Geoff Nichols
Labels: linux manage-user-group redmine type_and_provider
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Geoff Nichols (JIRA)

unread,
Mar 24, 2018, 7:41:05 PM3/24/18
to puppe...@googlegroups.com

Branan Riley (JIRA)

unread,
May 9, 2018, 2:14:03 PM5/9/18
to puppe...@googlegroups.com
Branan Riley updated an issue
Change By: Branan Riley
Labels: manage-user- autorequire group redmine type_and_provider user

Branan Riley (JIRA)

unread,
May 9, 2018, 2:15:05 PM5/9/18
to puppe...@googlegroups.com
Branan Riley updated an issue
Change By: Branan Riley
Labels: autorequire group redmine triaged type_and_provider user

Josh Cooper (JIRA)

unread,
May 22, 2019, 5:50:04 PM5/22/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-1153
 
Re: Incorrect ordering of group and user resource purging

There are a few different issues filed in this ticket:

1. Can't guarantee the order of user and group deletion. The original issue was filed when puppet did not support manifest ordering. Adding explicit dependencies to ensure resources were deleted in the correct order, didn't work due to PUP-2451. However, Puppet defaults to manifest ordering, so you can easily delete users before groups.

2. USERGROUPS_ENAB=yes/no. If the setting is yes, then ensuring the user and group are absent works correctly (as of 5.5.13) because the group provider determines whether the resource is insync after it's been deleted. So the provider sees that the group doesn't exist and nothing needs to be done:

root@fzt1mvzv8hybr6g:~# cat create.pp
group { 'foo':
  ensure => 'present',
}
user { 'foo':
  ensure => 'present',
  groups => 'foo',
}
root@fzt1mvzv8hybr6g:~# puppet apply create.pp
Notice: Compiled catalog for fzt1mvzv8hybr6g.delivery.puppetlabs.net in environment production in 0.01 seconds
Notice: /Stage[main]/Main/Group[foo]/ensure: created
Notice: /Stage[main]/Main/User[foo]/ensure: created
Notice: Applied catalog in 0.12 seconds
root@fzt1mvzv8hybr6g:~# cat delete.pp
user { 'foo':
  ensure => 'absent',
}
group { 'foo':
  ensure => 'absent',
}
root@fzt1mvzv8hybr6g:~# puppet apply delete.pp
Notice: Compiled catalog for fzt1mvzv8hybr6g.delivery.puppetlabs.net in environment production in 0.01 seconds
Notice: /Stage[main]/Main/User[foo]/ensure: removed
Notice: Applied catalog in 0.10 seconds
root@fzt1mvzv8hybr6g:~# puppet resource group foo
group { 'foo':
  ensure => 'absent',
}

3. Discovering users in a group: If the group can't be deleted until all of its users have been deleted, then you first have to know all of the users that need to be deleted. This is what the resources type and purging are intended to solve. However, the resources type does not understand about system groups, so it can try to delete the root, wheel, etc group if you're not careful.

4. Forcing the deletion of groups leaving users with dangling "pointers" seems like the wrong approach as Eric Sorenson and Charlie Sharpsteen have already mentioned.

Given the need to purge unmanaged users and groups, I think it would be best to fix the resources type so it doesn't delete system groups, for example, using unless_system_group. Ignoring the issue with system groups, purging does work as expected:

root@fzt1mvzv8hybr6g:~# cat create.pp
group { 'foo':
  ensure => 'present',
}
user { ['foo','bar','baz']:
  ensure => 'present',
  groups => 'foo',
}
root@fzt1mvzv8hybr6g:~# puppet apply create.pp
Notice: Compiled catalog for fzt1mvzv8hybr6g.delivery.puppetlabs.net in environment production in 0.01 seconds
Notice: /Stage[main]/Main/Group[foo]/ensure: created
Notice: /Stage[main]/Main/User[foo]/ensure: created
Notice: /Stage[main]/Main/User[bar]/ensure: created
Notice: /Stage[main]/Main/User[baz]/ensure: created
Notice: Applied catalog in 0.23 seconds
root@fzt1mvzv8hybr6g:~# cat resources.pp
resources { 'user':
  purge => true,
  unless_system_user => 500,
}
resources { 'group':
  purge => true,
  unless_system_user => 500,
}
root@fzt1mvzv8hybr6g:~# puppet apply resources.pp
Notice: Compiled catalog for fzt1mvzv8hybr6g.delivery.puppetlabs.net in environment production in 0.02 seconds
Notice: /Stage[main]/Main/User[foo]/ensure: removed
Notice: /Stage[main]/Main/User[bar]/ensure: removed
Notice: /Stage[main]/Main/User[baz]/ensure: removed
Notice: /Stage[main]/Main/Group[foo]/ensure: removed
Notice: Applied catalog in 0.24 seconds

Jesse Hathaway (JIRA)

unread,
May 24, 2019, 4:17:04 PM5/24/19
to puppe...@googlegroups.com

Josh Cooper We manage all our users, including system users, with puppet so your suggestion would not help our use case. We have been running a version of my PR for three years with no ill effects. I am not sure what the concerns are with "dangling pointers" the affected user can still login and even access any files, they just can't resolve their GID into a name. You are correct that through proper ordering you can delete the user before the group, but there are a couple of issues with that approach:

  1. This behavior is inconsistent between platforms, for example FreeBSD or DragonflyBSD will happily delete a group when there still exists a user who has that group set as their GID. This inconsistency makes it harder to use cross platform modules on the forge.
  2. You have to change your puppet ordering anytime you delete a user, since on creation you want the group created before the user and upon deletion you want the user deleted before the group:

    # Creation
    group { 'butter':
      ensure => present,
      gid => 555,
    }
     
    user { 'butter':
      ensure  => present,
      uid     => 555,
      gid     => 555,
      require => Group['butter'],
    }
     
    # Deletion
    user { 'butter':
      ensure  => present,
      uid     => 555,
      gid     => 555,
    }
     
    group { 'butter':
      ensure => present,
      gid => 555,
      require => User['butter'],
    }
    

Removing the safeguard has no effect in practice. Though I sympathize with the desire for puppet to properly order the removal, in practice that is cumbersome for the developer. The onus should be on Puppet to provide a richer abstraction, rather than asking Puppet developers to work around an annoyance which provides no discernible value.

Robert August Vincent II (JIRA)

unread,
Jun 3, 2019, 1:23:04 PM6/3/19
to puppe...@googlegroups.com

Note that the accounts module now adds groups before users, and deletes users before groups.

Josh Cooper (JIRA)

unread,
Jun 3, 2019, 7:24:05 PM6/3/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-1153

Jesse Hathaway if you manage all users including system users and groups, then I would expect the following to just work:

   user { 'root':
    ensure => present,
    password => ...,
    require => Group['wheel'],
  }
 
  # other users here
 
  group { 'wheel':
    ensure =>present,
  }
 
  # other groups here
 
  resources { 'user':
    purge => true,
  }
 
  resources { 'group':
    purge => true,
  }

There are several reasons why this didn't previously:

  • The manifest above doesn't have an explicit dependency between Resources['user'] and Resources['group']. Prior to manifest ordering, there was no guarantee about the order of evaluation, so puppet could attempt to purge groups before users.
  • Generated resources (such as recursive file resources or purged user/group/tidy) didn't have any relationships added to them. As a result, puppet could evaluate them in unexpected orders. In PUP-1963, puppet makes sure that generated resources are evaluated adjacent to the resource that generated them. This way the purged users are evaluated immediately after the Resources['user'] resource and before the Resources['group'] resource.

Alternatively, you can explicitly list the users/groups needing to be deleted. As mentioned earlier, that suffers from 1) having to discover which users/groups need to be deleted, and 2) the order of resources needs to be reversed when ensuring absent due to a long standing puppet bug. But if you put things in the right order it works as expected.

Given all of that, I don't believe forcing group deletion is necessary, instead just use the resources resource. But if I'm mistaken or there is an issue with resources please let me know.

Jesse Hathaway (JIRA)

unread,
Jun 7, 2019, 3:28:04 PM6/7/19
to puppe...@googlegroups.com

Josh Cooper You are correct that forcing group deletion is not necessary if I use the resources resource. That methodology should be sufficient for my use case, so I will go ahead and close my pull request. I maintain that this is a relatively innocuous change which improves usability and eliminates a difference between Linux and platforms such as FreeBSD, but evidently you disagree on the importance of those factors.

Josh Cooper (JIRA)

unread,
Jun 10, 2019, 1:20:04 PM6/10/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-1153

To watchers on this ticket. The root cause for this issue was due to (PUP-1963) generated resources not having any relationships added to them, which could cause things to be executed in incorrect order. This caused issues for any provider using Provider#generate or Provider#eval_generate to generate new resources at catalog application time.

Jesse Hathaway in situations were we need to force behavior, we generally add a force parameter to the type and default it to false. For example, when deleting a directory with recurse true. I think that approach might work here, if you want to force group deletion.

 

Reply all
Reply to author
Forward
0 new messages