Puppet NetApp Network device management...

[Gavin Williams]

Hi all

Following on from my previous post Puppet Network Devices.. I've now started work on creating a NetApp network device provider for Puppet...

I'm trying to start simply by just gathering some facts from our filers.

The code soo far is available here: https://github.com/fatmcgav/shrug-netapp/ Credit to the original author shrug for setting out the framework...

Anyhow, onto my first challenge.

I've downloaded and added the relevant NetApp SDK files into modules/shrug-netapp/lib/puppet/util/network_device/netapp/ as shown below:

$ ls -l modules/shrug-netapp/lib/puppet/util/network_device/netapp/
total 88
-rw-r--r--. 1 puppet puppet   924 Nov  2 17:13 device.rb
-rw-r--r--. 1 puppet puppet  8229 Nov  2 17:16 DfmErrno.rb
-rw-r--r--. 1 puppet puppet   228 Nov  2 17:00 facts.rb
-rw-r--r--. 1 puppet puppet  9451 Nov  2 17:16 NaElement.rb
-rw-r--r--. 1 puppet puppet 36089 Nov  2 17:15 NaErrno.rb
-rw-r--r--. 1 puppet puppet 19036 Nov  2 17:15 NaServer.rb

 
The NaServer class is then included within the device.rb file above.
However Puppet is currently failing with:

Info: starting applying configuration to actint-star-nactl01 at actint-star-nactl01
Error: Could not run: no such file to load -- NaElement

This NaElement file is require'd by the NaServer class.
However it seems to be failing due to the include path not including the NaElement.rb file...
Therefore I'm guessing that Puppet/Ruby is looking for the NaElement file in some default location, rather than the same location as the NaServer class...

I've generated the same error on some tests scripts by moving the NetApp SDK files into a sdk sub folder, and then just "require sdk/NaServer" in my script, which generates a very similar error:

ruby hello_ontapi.rb  192.168.35.118 root xxx
./sdk/NaServer.rb:21:in `require': no such file to load -- NaElement (LoadError)
        from ./sdk/NaServer.rb:21
        from hello_ontapi.rb:24:in `require'
        from hello_ontapi.rb:24

Any ideas how I could work around this?

Cheers
Gavin
 

[Gavin Williams]

Ok, after a bit more googling, have solved the NaElement load error...

Tweaked the NaServer.rb file as follows:

#require 'NaElement'
require File.dirname(__FILE__) + "/NaElement"

I've now successfully loaded the require Netapp SDK files, and started working through the connection process...
Though it looks like it's hit another error:

Debug: Puppet::Device::Netapp: connecting to Netapp device actint-star-nactl01.
Debug: Puppet::Device::Netapp: configdir is .
Error: Can't load netapp for actint-star-nactl01: undefined method `+' for nil:NilClass

Tracked the above back to device.rb, where it's attempting to use 'Puppet[:configdir]' but is failing to find the value...

So now to work out what's going on there :)

Cheers
Gav

[Gavin Williams]

Ok, have fixed that error aswell...

Replaced ':configdir' with ':confdir'.
However it's now failing due to the netapp.yml file not being present... What's confused me is the path it's come up with for ':confdir'. According to the debug output, it's trying to use

ESC[0;36mDebugESC[0m: Puppet::Device::Netapp: connecting to Netapp device actint-star-nactl01.
ESC[0;36mDebugESC[0m: Puppet::Device::Netapp: configdir is /var/lib/puppet/devices/actint-star-nactl01.

However I would prefer it to use /etc/puppet on the master... 
Any way I could do this, other than hard-coding?

Cheers
Gavin

[Gavin Williams]

Me again...

I pressed on using the /var/lib/puppet... configdir for the time-being, and with a bit more jiggery pokery, I've managed to successfully register a NetApp device, and extract some facts from it... And here's the photographic proof :) :D http://ow.ly/i/15SLN

Now to start looking at how to handle the next steps, such as volume creation, quota creation, export creation... :D

Latest code has been pushed to Github...

Still need to work out how to use a central confdir rather than agent specific ones, but can come back to that :)

Cheers
Gavin

[Gavin Williams]

Ok, moving on then...

I've created a 'netapp_volume' type and provider, and am trying to test it against one of the existing NetApp device nodes:
manifests/site.pp contains:

node 'actint-star-nactl01' {

        # Test NetApp volume provider
        netapp_volume { 'v_puppet_test':
                ensure => present,
        }

}

When running 'puppet device -v', I get:

$ sudo puppet device -v

Info: starting applying configuration to actint-star-nactl01 at actint-star-nactl01

Warning: Local environment: "production" doesn't match server specified node environment "development", switching agent to "development".
Info: Retrieving plugin
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/type/netapp_volume.rb]/content: content changed '{md5}2f3e20a3878f9ed94a5fffc4a9811ef7' to '{md5}ebe14c73783c97063d9b6ae0bc791896'
Error: Could not autoload puppet/provider/netapp_volume/netapp_volume: uninitialized constant Puppet::Provider::Netapp
Error: Could not autoload puppet/type/netapp_volume: Could not autoload puppet/provider/netapp_volume/netapp_volume: uninitialized constant Puppet::Provider::Netapp
Error: Could not retrieve catalog from remote server: Could not intern from pson: Could not autoload puppet/type/netapp_volume: Could not autoload puppet/provider/netapp_volume/netapp_volume: uninitialized constant Puppet::Provider::Netapp
Using cached catalog
Info: Applying configuration version '1352132516'
Error: /Stage[main]//Node[actint-star-nactl01]/Netapp_volume[v_puppet_test]: Could not evaluate: No ability to determine if netapp_volume exists
Finished catalog run in 0.21 seconds

So it's failing to autoload my new provider, however I'm not sure what 'uninitialised constant' means :s
The class Puppet::Provider::Netapp is defined in lib/puppet/provider/netapp.rb.

I've pushed the latest edits into Github.

Ideas welcome.

Cheers
Gavin

[Gavin Williams]

Looks like the below error was caused by some locally cached code that hadn't been updated by my Git pull :s

Resolved that, and now am getting the following:

$ sudo puppet device -v
Info: starting applying configuration to actint-star-nactl01 at actint-star-nactl01
Warning: Local environment: "production" doesn't match server specified node environment "development", switching agent to "development".
Info: Retrieving plugin

/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet]/mode: mode changed '0775' to '0755'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/provider]/mode: mode changed '0775' to '0755'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/provider/netapp.rb]/content: content changed '{md5}399796914d8aeb68387455de5b24f735' to '{md5}3be2121ab99116c4bc89db21d5d4b54b'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/provider/netapp_volume/netapp_volume.rb]/content: content changed '{md5}46b2a0004e0245c577974e7a645e08f7' to '{md5}1b06628868826a8ea64744a7ae8ec8da'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/type]/mode: mode changed '0775' to '0755'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/type/netapp_volume.rb]/content: content changed '{md5}ebe14c73783c97063d9b6ae0bc791896' to '{md5}2fb244482c7d760e6cc132f104be619d'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/util/network_device/netapp/facts.rb]/content: content changed '{md5}c7e59e79e19486cba9ae7e5314b2a061' to '{md5}b0c912dc663d5251219bbad895455834'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/util/network_device/netapp/device.rb]/content: content changed '{md5}7354a748bd41582fcf76bf2643098fcd' to '{md5}9d2e26f97c7ea6ea16177acb8df85714'
/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/util/network_device/netapp/NaServer.rb]/content: content changed '{md5}cd815964f9a2dfe757f5365a848b80d5' to '{md5}63f12abed9d1145b6d326c9fc06eb0c0'
Info: Caching catalog for actint-star-nactl01

Info: Applying configuration version '1352132516'

Error: /Stage[main]//Node[actint-star-nactl01]/Netapp_volume[v_puppet_test]: Could not evaluate: undefined method `transport' for #<Puppet::Util::NetworkDevice::Netapp::Device:0x7ff3142c80f8>
Finished catalog run in 0.11 seconds

If I change provider/netapp_volume/netapp_volume.rb as follows:

/File[/var/lib/puppet/lib/puppet/provider/netapp_volume/netapp_volume.rb]/content:
--- /var/lib/puppet/lib/puppet/provider/netapp_volume/netapp_volume.rb  2012-11-05 17:26:56.257616498 +0000
+++ /tmp/puppet-file20121105-8521-10tepn6-0     2012-11-05 17:40:16.440612301 +0000
@@ -19,7 +19,7 @@

   def exists?
     Puppet.debug("Puppet::Provider::Netapp_volume: checking existance of Netapp Volume #{resource[:name]}")
-    transport.invoke("volume-list-info").include?(resource[:name])
+    @transport.invoke("volume-list-info").include?(resource[:name])
   end

I get:

 Info: starting applying configuration to actint-star-nactl01 at actint-star-nactl01
Warning: Local environment: "production" doesn't match server specified node environment "development", switching agent to "development".
Info: Retrieving plugin

/File[/var/lib/puppet/devices/actint-star-nactl01/lib/puppet/provider/netapp_volume/netapp_volume.rb]/content: content changed '{md5}1b06628868826a8ea64744a7ae8ec8da' to '{md5}feb535835cefd0921c8be5d0af6e11f2'
Info: Caching catalog for actint-star-nactl01

Info: Applying configuration version '1352132516'

Error: /Stage[main]//Node[actint-star-nactl01]/Netapp_volume[v_puppet_test]: Could not evaluate: undefined method `invoke' for nil:NilClass
Finished catalog run in 0.20 seconds

So not sure why it's not working either way...

Any ideas???

Cheers
Gavin 

 

[Gavin Williams]

Ok, quick update...

I've not managed to get past the 'Undefined method' error despite my best efforts today...

Any ideas on where I could be going wrong?

Cheers
Gavin

[Gavin Williams]

Scratch my last, had a bit of a eureka moment...

Added ':transport: to the attr_accessor against device.rb, and it started working again...

However now failing elsewhere...
Have got the following against a relevant node:

        # Test NetApp volume provider
        netapp_volume { 'v_puppet_test':
                ensure => present,

                initsize => "1g",
                aggr => "aggr01",
                spaceresv => "none",
        }

The run is failing with:

 Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid parameter aggr at /etc/puppet/manifests/site.pp:30 on node actint-star-nactl01
Error: Could not set 'present' on ensure: undefined method `result_reason' for #<NaElement:0x7ffb9cb5d160> at 27:/etc/puppet/manifests/site.pp
Error: Could not set 'present' on ensure: undefined method `result_reason' for #<NaElement:0x7ffb9cb5d160> at 27:/etc/puppet/manifests/site.pp
Wrapped exception:
undefined method `result_reason' for #<NaElement:0x7ffb9cb5d160>
Error: /Stage[main]//Node[actint-star-nactl01]/Netapp_volume[v_puppet_test]/ensure: change from absent to present failed: Could not set 'present' on ensure: undefined method `result_reason' for #<NaElement:0x7ffb9cb5d160> at 27:/etc/puppet/manifests/site.pp

 Got some debug logging, which is giving:

^[[0;36mDebug^[[0m: Puppet::Provider::Netapp_volume: checking existance of Netapp Volume v_puppet_test
^[[0;36mDebug^[[0m: Puppet::Provider::Netapp_volume: Vol Info: <results status="failed" reason="No volume named 'v_puppet_test' exists" errno="13040"></results>

^[[0;36mDebug^[[0m: Puppet::Provider::Netapp_volume: Volume doesn't currently exist.
^[[0;36mDebug^[[0m: Puppet::Provider::Netapp_volume: creating Netapp Volume v_puppet_test of initial size 1g

So I can see that the provider exists? is working correctly, and it's trying to fire the create def...

The type is defined correctly afaik:

Puppet::Type.newtype(:netapp_volume) do 
  @doc = "Manage Netapp Volume creation, modification and deletion."
  
  apply_to_device
  
  ensurable do
    desc "Netapp Volume resource state. Valid values are: present, absent."
    
    defaultto(:present)
    
    newvalue(:present) do 
      provider.create
    end
    
    newvalue(:absent) do 
      provider.destroy
    end
  end
  
  newparam(:name) do
    desc "The volume name."
    isnamevar
    #newvalues(/^[[:alpha:][:digit:]\.]+$/)
  end

  newparam(:initsize) do
    desc "The initial volume size."
    defaultto "1g"
     
  end
  
  newparam(:aggr) do
    desc "The aggregate this volume should be created in." 
    
  end
  
  newparam(:spaceresv) do
    desc "The space reservation mode."
    
  end
end

Latest code in Git... 

Any ideas?

Cheers
Gavin

On Monday, 5 November 2012 17:46:07 UTC, Gavin Williams wrote:

[Gavin Williams]

Quick update... It looks like all that was needed to resolve the below was a puppetmaster restart.

Have successfully managed to create a new volume on the NetApp filer using puppet :) :D

Now to plough on with adding support for the rest of the bits we need :)

Cheers
Gavin

[Gavin Williams]

:( Seems to be one step forward and two steps back atm...

Following the successful creation and deletion of NetApp filer volumes using puppet earlier in the day, I added the required provider and type to support qtree creation and deletion...

However since then, I haven't been able to create either a new volume or a new qtree reliably...
It seems that it will intermittently work, but the majority of the time the puppet device run will fail with

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not autoload puppet/type/netapp_volume: Could not autoload puppet/provider/netapp_volume/netapp_volume: uninitialized constant Puppet::Util::NetworkDevice on node actint-star-nactl01

This error alternates between provider/netapp_volume/netapp_volume and provider/netapp_qtree/netapp_qtree...
It doesn't appear to be due to config, as I've stripped the node config right back to base and still seeing errors...

The node config I was trying to work towards is:

        netapp_qtree { 'q_puppet_test1':
                ensure => present,
                volume => 'v_puppet_test',
                require => Netapp_volume['v_puppet_test1']
        }

        netapp_volume { 'v_puppet_test1':
                ensure => present,
                initsize => "1t",
                aggregate => "aggr01",
                spaceres => "none",
        }

I've tried multiple agent runs, multiple master restarts, all to no avail...

Any ideas on what could be the issue?

As always, latest code is available in Git...

Cheers
Gavin

[Gavin Williams]

Quick update on the below error...

I dropped into the #puppet-dev IRC channel, and was able to resolve the issue with some assistance from Dominic, so many thanks for that...

Latest code has been committed to Git, but basically I was missing a couple of requires for Puppet libs...

Now happily creating volumes and qtrees in the same run.
Onwards to the next functionality requirement now... Exports :D

Cheers
Gav

[Gavin Williams]

Monday update from me...

Implemented support for Export creation on Friday, after making some further good progress...

Now I'm looking at fine-tuning stuff, such as volume settings, export options, etc...

Cheers
Gavin

[Jakov Sosic]

On 11/12/2012 12:59 PM, Gavin Williams wrote:
> Monday update from me...
>
> Implemented support for Export creation on Friday, after making some
> further good progress...
>
> Now I'm looking at fine-tuning stuff, such as volume settings, export
> options, etc...

Great work! When I get some spare time I will continue to work on my
cobbler custom types, so I will take a peek into your code aswell. My
providers are good for our internal use so I pushed on with other stuff
although they still need some optimizations. Just keep on the good work :)

--
Jakov Sosic
www.srce.unizg.hr

[fatmcgav]

Jakov

Thank you for the positive feedback.

The NetApp puppet stuff is on the back burner atm, hopefully should be able to get back to buy in the new year :-)

Cheers
Gav

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.