Exists? is called before prefetching

[bert hajee]

Hallo,

I'm using the puppet module transition to help me out a nasty puppet definition situation. But I noticed it sometimes is not idempotent. We noticed this on a very complex custom type. To make sure the issue is a clear as can be, we extracted the minimal type/and provider to simulate this. Here you can find the type/provider code 

Here is some simple example code, based on a simple custom type:

  transition { 'transition':

    resource   => File['/a.a'],

    attributes => {

      content  => 'temp',

    },

    prior_to   =>  Error_type['a']

  }

file{'/a.a':

  ensure => 'present',

  content => 'aa',

}

error_type {'a':

  ensure => 'present',

  prop => 'aaa'

}

Using this example code, I noticed that the exists? is called before prefetching is done. Before using this module, I was under the impression the exists? method was only called later in the sequence. Therefore my exists? method was based on prefetched information.

Do I:

• register a bug/question in the module?
• change the type/provider implementation to check if the specific resource is already prefetched (would probably need to be done for a lot more types/and providers)
• register a bug/question for puppet, making sure the prefetch is ALWAYS called before exists?

All suggestions are welcome!

Regards,

Bert 

[Trevor Vaughan]

HI Bert,

Exists should be checking the @property_hash object which is populated by the instances method.

Something like:

def exists?

  @property_hash[:ensure] == :present

end

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/94724428-5db3-4409-b710-9e2847bd1c47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[Trevor Vaughan]

Sorry for the double post but this is probably the best walkthrough on the subject out there presently http://garylarizza.com/blog/2013/12/15/seriously-what-is-this-provider-doing/

[Reid Vandewiele]

I haven't dived into the code recently but depending on when prefetching happens, it might be possible the Transition type is causing an "early" invocation of #exists?. This is because Transition invokes a check of the resource it is "prior to", thusly: https://github.com/puppetlabs/puppetlabs-transition/blob/0.1.1/lib/puppet/provider/transition/ruby.rb#L68

If it's the case that prefetch isn't called until the first instance of a given type is evaluated, that might be something that's happening *after* the Transition resource does its thing. Which could help explain why your exists?() method, which uses prefetched data, isn't working.

If, however, prefetch is called for ALL providers before ANY resources are evaluated, you can ignore everything else I've said above because it's all speculation from a false premise. :)

No solutions here, just largely unresearched speculation, but maybe some useful thoughts. :)

~Reid

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/94724428-5db3-4409-b710-9e2847bd1c47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[bert hajee]

Trevor, Reid,

Thanks for taking the time to look at this. 

Exists should be checking the @property_hash object which is populated by the instances method.
Something like:
def exists?
  @property_hash[:ensure] == :present
end

The real type does implement the exists? method by looking at the property hash. But when using the transition type, the property hash is not yet filled correctly, because the instances and prefetched methods are not (yet) called.

On Friday, 14 July 2017 22:57:54 UTC+2, Reid Vandewiele wrote:

I haven't dived into the code recently but depending on when prefetching happens, it might be possible the Transition type is causing an "early" invocation of #exists?. This is because Transition invokes a check of the resource it is "prior to", thusly: https://github.com/puppetlabs/puppetlabs-transition/blob/0.1.1/lib/puppet/provider/transition/ruby.rb#L68

If it's the case that prefetch isn't called until the first instance of a given type is evaluated, that might be something that's happening *after* the Transition resource does its thing. Which could help explain why your exists?() method, which uses prefetched data, isn't working.

 

 This seems to b what is happening.

 

But what is the best way forward to solve this?

[Reid Vandewiele]

This could be seen as a problem with Transition, which would be best to see fixed in the Transition module. The basic idea would be to make sure a provider has done any prefetching it needs to before calling safe_insync?() on a resource. That module is effectively opensource though so it depends on someone being aware of the problem and having spare time to fix it.

Since that may not happen quickly, a workaround might be to ensure that at least one instance of the type in question is evaluated prior to the transition resource being evaluated. E.g.

file { 'prefetch':

  path   => '/dev/null',

  ensure => present,

}

transition { 'transition a file resource':

  require    => File['prefetch'],

  resource   => File['/path/to/real/file'],

  attributes => { ensure => absent },

  prior_to   => Service['example'],

}

This is just a mock example. The only thing it's really showing is that a file "resource" called prefetch exists, doesn't really do anything, but is guaranteed to be evaluated before a transition involving a different, real file.

[bert hajee]

Reid,

Thanks for this direction. I will try and see if I can come up with a good PR to solve this.