Custom type, relationship and fetch order

[Thomas Champagne]

Hi,

I try to develop a new module to deploy resources with an API REST. But I have problems to define relationship between two resources.
I haven't problem to fetch and flush resource individually. This two resources are fetch with a REST endpoint but I can have the relationship between resources with only one endpoint. When I want to delete a resource, I must remove the relationship before removing this resource. But I can't know its dependencies because it is not returned by its REST endpoint. They are returned by the endpoint of the other resource.

It is not easy to describe it so I'm going to explain it with an example.
I have two resources group and user with two REST endpoints (with an example response) :
http://host/api/group :

[{name: 'developer'},{name: 'user'}]

http://host/api/user :

[{
    name: 'bob',
    groups: ['developer', 'user']
}, {
    name: 'alex',
    groups: ['user']
}]

So I create two types and two provider to manage them :

Puppet::Type.newtype(:group) do
    apply_to_device
    ensurable
    newparam(:name, :namevar => true)
end
Puppet::Type.newtype(:user) do
    apply_to_device
    ensurable
    newparam(:name, :namevar => true)
    newproperty(:groups, :array_matching => :all)
    autorequire(:group) do
        self[:group]
    end
end

The puppet config is :

node 'host' {
    user { 'bob':
        ensure => present,
        groups => ['developer', 'user'],
    }
    user { 'alex':
        ensure => present,
        groups => ['user'],
    }
    group { 'developer',
        ensure => present,
    }
    group { 'user',
        ensure => present,
    }
}

When I run "puppet device", the log is :
prefetch group
flush group 'developer'
flush group 'user'
prefetch user
flush user 'bob'
flush user 'alex'

Now I want to delete a group :

node 'host' {
    user { 'bob':
        ensure => present,
        groups => ['user'],
    }
    user { 'alex':
        ensure => present,
        groups => ['user'],
    }
    group { 'developer',
        ensure => absent,
    }
    group { 'user',
        ensure => present,
    }
}

When I run "puppet device", the log should be :
prefetch group
destroy group 'developer'
prefetch user
flush user 'bob'

But, when I want to delete a group, the endpoint check if is not affected to a user and throw an exception it there is any relationship. So, I must remove all user group before remove the group.
But I think this is not possible to do this with puppet because the management of each resource is by type. Puppet doesn't know the relations of a group when it flush the data group.
Is it possible with puppet to update the state of resources in this order :
prefetch group
prefetch user
destroy group 'developer' and relationship with user 'bob'
flush user 'bob'

I try other solution with an autosubscribe instead of autorequire to have a notification when the group is deleted but this notification happens at the end :
prefetch group
destroy group 'developer'
flush group 'user'
prefetch user
flush user 'bob'
flush user 'alex'
notify user 'bob'

I think the best order to do this is :
prefetch group
prefetch user
notify user 'bob' => Call an REST endpoint to remove relationship
destroy group 'developer'
flush group 'user'
flush user 'bob'
flush user 'alex'

Cheers,
Thomas

[Felix Frank]

Hi,

I believe that all you have to do is taking care of the proper resource dependencies.

If you really want to rely on autorequire for this, you will have to make the logic more dynamic. This is easier if you target a Puppet version that has support for autobefore in addition to autorequire.

1. Have the user autorequire all groups if ensure => present
2. Have the user autobefore all groups is ensure => absent

This way, Puppet will make sure to remove all users before managing the group.

Of course, you can do this more safely by adding the appropriate require/before parameters in your manifest.

HTH,
Felix

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/df12fe71-070e-4a37-8b6f-e264b9470daf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Champagne]

Thanks for answering Felix,

I solve some problems with require/before parameters in the manifest. I create a new type (user_group) that define link between user and group. The provider of user_group type call the provider of user to get the relationship.

And I define new type in the manifest module :

define rest::user (
  $user_name = $title,
  $ensure = present,
  $groups = [],
) {
  user { $title: ensure => $ensure, before => Group[$groups], }
 user_group { $title: ensure => $ensure, groups => $groups, require => [User[$title], Group[$groups]], }
}

define rest::group (
  $user_name = $title,
  $ensure = present,
) {
  group { $title: ensure => $ensure, }
}


So in the manifest, I define my node like this :


node 'host' {
    rest::user { 'bob':
        ensure => present,
        groups => ['user','developer'],
    }
    rest::user { 'alex':

        ensure => present,
        groups => ['user'],
    }

    rest::group { 'developer',
        ensure => present,
    }
    rest::group { 'user',
        ensure => present,
    }
}

Now, the log is


prefetch user
flush user 'bob'
flush user 'alex'
prefetch group
flush group 'developer'
flush group 'user'


prefetch user_group
flush user_group 'bob'
flush user_group 'alex'

I have last problem. When I only configure group that ensure is absent :

node 'host' {
  rest::group { 'developer',
    ensure => absent,
  }
}

There is no user defined and so, user provider doesn't fetch user data. If there is a real relationship between the group and a user, the deletion failed because group provider doesn't know the relationship.
Is there a way to force fetch of a type ?

Thomas

[Felix Frank]

Hi,

I'm not sure about directly forcing a prefetch action.

However, it seems to me that it would make sense for you to implement a *generate* method for your group type. It should enumerate the users that are members of the group and create a Puppet resource each. (Don't worry about duplicates, Puppet will prune those for you.)

HTH,
Felix

[Thomas Champagne]

Yes, I agree with you. I have seen this solution in the "resources" type that generate resources to purge them.
So, you tell : in the generate method of "group" type, it call instance method of the user provider to generate user. With this solution, if I define a user in the node, the rest endpoint will be apply twice : the first time with prefetch "user" provider and second time by the generate method of "group" type. I would like to limit calls to REST API.

I think I must cache answer of the first call to rest endpoint user in the "user" provider. So, the "group" provider can get relationship without calling again the REST API. If there is no defined user, "user" provider doesn't have prefetched data but it will cache them at first call.

Do you think it is the better implementation ?
Do you know if there is other module with this implementation ?