Re: QUERY : no resources for remote admin?

[Jenner La Fave]

Puppet is not really designed to do what you're asking. The idiomatic Puppet way to install ksh on all your nodes would be to create a node declaration for each host and either add Package[ksh] on each node or have them include a class with that resource definition.

If you're looking for a way to push but not manage configurations remotely, you might want to try scripting with MCollective or something like it.

-- Jenner

On Wednesday, September 5, 2012 12:30:05 PM UTC-7, am-aaron wrote:

hello fellow engineers:

i am currently developing a Puppet manifest to Deploy (Install, Configure) a Software Package onto a number of Nodes from a Machine that is network connected to the Nodes (can ssh). the Yum Repo for the Nodes is on this Deployment Point.

it was and still is very surprising to find that Puppet does not have a Resource to allow a Remote Package installation to be done. Package{} works on the current Node and will use yum by default to install a Software Package. however, if i want to do a yum install from another Deployment Point, Puppet does not offer a Resource.

Local Package Install:

    package { 'ksh':

        ensure => installed

    }

i also found out by simply trying that i can do a installation using lists as in (also found on Puppet Cookbook much later):

    package { [  'ksh', 'mksh' ]:

        ensure => installed

    }

but, if i want to install ksh onto Node node from Machine master, i need to use Exec:

    # Install package

    exec { 'remote-install':

      command   => "/usr/bin/ssh root@node 'yum -y install ksh'",

      returns   => 0,

      logoutput => on_failure,

    }

what does Puppet Labs and the User Community recommend? is it better and more efficient to create a second Puppet manifest for the Nodes so that i can use Package{}? that would mean:

* additional Puppet manifests for the Nodes in addition to the manifest on the Deployment Point, which is used for configuration;

* additional communication between the Puppet Agent and Puppet master for package installation (?);

* additional overhead of maintaining two Puppet manifests;

* additional overhead of synchronizing the Installation on Nodes with the Configuration on Deployment Point.

why does PuppetLabs simply not offer simple Resources to do what we require here? is there some Puppet ic way of designing and coding manifests that i do not know about?

thank you very, very much in advance.

ciao,

Aaron

--

{ celltext Nokia E90 Communicator: +353-89-420-8033; Skype: am-aaron }

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.

[am-aaron]

hello Jenner:

thank you.

without intending to be rude or scornful: using MCollective? hmmm. instead, i can use a simple shell script, and it works great in minutes. Puppet makes us go bonkers and i do not think its really intuitive to understand. instead a procedural scipt is just great.

i find this restriction very strange, though i understood recently that you use Puppet to declare how a system should be.

i have a problem: i want to install a software package on multiple nodes and then have to run a configure program to configure all the nodes. its not distributing configuration files, i have to run the program only on 1 of the nodes or from a separate machine. its not easy to synchronize all the nodes and to ensure that all machines are in the same state before you run the configure.

instead, it damn easy to run the Install and Configure from a single machine to all the Nodes. no synchronization problems and one manifest to maintain.

Puppet also has big issues with this scenario: from one machine to check the status of a file on another machine and to then take action. the File{} resource only applies to the local machine. so, how do i do distributed system admin or remote sys admin?

i spend so much time to learn Puppet, we pay a good deal for it, and we spend some time to understand how to use the declarative language, but i cannot do simple things. even doing an 'echo' to log output is so complicated. if we have to resort to using Ruby, its even more complicated. enhances my qualifications, but is it that vital to use it?

genuinely puzzled,

Aaron

[Jenner La Fave]

Like I said, these aren't "issues" with Puppet, you're just using a screwdriver to hit a nail. All resources only apply to the local machine. Your manifests are supposed to declaratively describe what the state and configuration of a node should be, and the Puppet agent makes it so after compiling the node's catalog.

Without knowing the details of the software you are trying to deploy, I would say either fix your deployment procedure so that Puppet can manage the configuration files on each node (with the typical Package, File, Service), or don't use Puppet at all.

-- Jenner

[jcbollinger]

On Wednesday, September 5, 2012 4:35:45 PM UTC-5, am-aaron wrote:

hello Jenner:

thank you.

without intending to be rude or scornful: using MCollective?

Meaning that you will be at least scornful, and we're supposed to let it slide?  I'd rather you just be scornful than disingenuous.

 

hmmm. instead, i can use a simple shell script, and it works great in minutes. Puppet makes us go bonkers and i do not think its really intuitive to understand. instead a procedural scipt is just great.

If a procedural script does the job satisfactorily for you, then why are you trying to make Puppet do it?

Puppet does not do orchestration.  It was not designed to do, and as far as I know there are no plans to introduce orchestration features into any future version.  It is not Puppet's problem that it is the wrong tool for your job.

With that said, if you want an automated orchestration solution, then there are several available, one of them being PuppetLabs's own MCollective.

As far as Puppet being easy to understand, you really have to look at it from the correct perspective.  Your description of what you want Puppet to do ("install", "run", "configure") suggests that you're focusing on the wrong things.  Puppet is not a scripting engine, or anything like one, so if you focus on actions you want to perform then you will indeed have a tough time of it.  Instead, Puppet is a state management system, so to understand it you need to focus on the system state you want to achieve and maintain.  Generally speaking, the actions performed if any to transition from the current state to the desired state are incidental to Puppet, at least from the view of the manifest author.

So if you find yourself thinking about Puppet in terms of verbs (i.e. "Puppet should [perform some action]") then you are probably looking at it the wrong way.  You should instead be thinking in terms of adjectives: "present", "installed", "running", etc..  Also, if you're looking to use Puppet to apply some change to your systems that you're not afterward going to maintain via Puppet, then you are probably trying to use the wrong tool.
 

i find this restriction very strange, though i understood recently that you use Puppet to declare how a system should be.

Your issue in fact follows directly from Puppet's nature.  It is because Puppet focuses on state (of individual nodes) instead of on transitions that it is not much good for orchestration.  Orchestration is all about coordinating actions.

 

i have a problem: i want to install a software package on multiple nodes and then have to run a configure program to configure all the nodes. its not distributing configuration files, i have to run the program only on 1 of the nodes or from a separate machine. its not easy to synchronize all the nodes and to ensure that all machines are in the same state before you run the configure.

Indeed, it is very difficult to do that with Puppet.  There's an old joke about a man who goes to the doctor and complains, "Doc, it hurts when I do <this>."  Says the doctor: "Then don't do that!"

I can think of at least one approach to achieve your specific goal via Puppet, but it would be complicated and ugly.  Don't do that!
 

instead, it damn easy to run the Install and Configure from a single machine to all the Nodes. no synchronization problems and one manifest to maintain.

And afterward, to avoid needlessly (or wrongly) doing that again?

 

Puppet also has big issues with this scenario: from one machine to check the status of a file on another machine and to then take action. the File{} resource only applies to the local machine.

Exported resources can be used to achieve some objectives of this kind.

 

so, how do i do distributed system admin or remote sys admin?

As I usually interpret those terms, that is exactly what MCollective and similar tools are for.  Or 'ssh -c' if you want to be minimalistic.
 

i spend so much time to learn Puppet, we pay a good deal for it, and we spend some time to understand how to use the declarative language, but i cannot do simple things.

Again, you are approaching Puppet the wrong way.  Most system administration tasks have this general form:

1. Check whether the target system is in the correct state with respect to <foo>
2. If not, then take the appropriate action to achieve the desired state
3. Some time later, go back to 1
   

Puppet allows (in fact demands that) you focus on the target state in your manifests.  That is, you describe where you want to be, not how you want to get there.  Puppet is pretty good at figuring out the latter for itself.

 

even doing an 'echo' to log output is so complicated.

Once more, "doing [...]."  If you want a custom message to appear in the Puppet's agent's log, then that's what Notify resources represent.

I hesitate even to mention it, as I am concerned that you will make trouble for yourself, but ultimately, Puppet's Exec resource allows it to do absolutely anything for which it has sufficient privilege.  Don't shoot too many toes off.

 

if we have to resort to using Ruby, its even more complicated. enhances my qualifications, but is it that vital to use it?

Of course not.  You should always choose the right tool for the job, and there are plenty of jobs for which that tool is not Puppet.  On the other hand, there are many, many jobs for which Puppet is a great tool.  If you're learning it merely to enhance your qualifications then you are completely missing the boat.  You should be using it because (and where) it makes your job and life easier.


John

[am-aaron]

good afternoon Jenner and JC Bollinger:

the question is connected to deployment (installation and configuration) of VCS. since VCS is quite popular in the indiustry, this will help many people.

thank you Jenner and JCB for your informative responses. i apologize because my initial response to Jenner was not really nice. i was stressed due to the deadline. i Normally do not use ¬that language and tone¬.

initially, i hesitated to explain which software had to be deployed, but now, here goes:

- we want to deploy Veritas Cluster Server (high-availability cluster software) onto multiple RHEL Nodes from a separate RHEL management node.

- Puppet Master runs on the Manager and each Node runs Puppet Agent.

- VCS software is a collection of RPMs with a separate installer executable.

- software can be installed using the Installer that uses SSH to install the RPMs onto multiple nodes, given the list.

- software can also be installed by simply installing the RPMs directly on each Node.

- the VCS RPMs have to be installed in a specifc order.

- once the RPMs are installed, a second Installer program is available on each Node, allowing Cluster Creation from one of the Nodes.

Questions

- which Puppet resource can we use to run the VCS software installer executable from the Puppet Master?

  the list of Nodes is to be passed to the Installer.

- if we want to install the VCS RPMs directly ourselves onto multiple Nodes, we can:

  - use Puppet 'package' on each Node or

  - use 'exec' on the manager to ssh and yum RPMs to multiple Nodes.

  is there a better way to use Puppet for this?

- VCS comes with:

  - an Installer to install VCS software onto all Nodes;

  - an Config program to create the Cluster;

  - CLI commands to create resources to be monitored.

  so, i think that it would be best to use Exec to run these programs.

finally, we are also considering the use of MCollective.

thank you,

Aaron

[jcbollinger]

On Tuesday, August 27, 2013 10:05:09 AM UTC-5, am-aaron wrote:

good afternoon Jenner and JC Bollinger:

the question is connected to deployment (installation and configuration) of VCS. since VCS is quite popular in the indiustry, this will help many people.

thank you Jenner and JCB for your informative responses. i apologize because my initial response to Jenner was not really nice. i was stressed due to the deadline. i Normally do not use ¬that language and tone¬.

initially, i hesitated to explain which software had to be deployed, but now, here goes:

- we want to deploy Veritas Cluster Server (high-availability cluster software) onto multiple RHEL Nodes from a separate RHEL management node.

- Puppet Master runs on the Manager and each Node runs Puppet Agent.

- VCS software is a collection of RPMs with a separate installer executable.

- software can be installed using the Installer that uses SSH to install the RPMs onto multiple nodes, given the list.

- software can also be installed by simply installing the RPMs directly on each Node.

- the VCS RPMs have to be installed in a specifc order.

- once the RPMs are installed, a second Installer program is available on each Node, allowing Cluster Creation from one of the Nodes.

Questions

- which Puppet resource can we use to run the VCS software installer executable from the Puppet Master?

  the list of Nodes is to be passed to the Installer.

None*.  The master just records resources in a catalog, which it then passes on to the client to apply.  You can, however, use the generate() function to run a program on the master during catalog compilation.  Your node list would need to be hardcoded into the appropriate manifest, loaded from some external source (e.g. via Hiera), or fed in by an external node classifier (ENC).

[*] Of course, the master can be its own client.  In that context, you could use an Exec resource to run the installer.

Inasmuch as this seems to be a singular, once-per-cluster activity, it is probably not something best done via Puppet.
 

- if we want to install the VCS RPMs directly ourselves onto multiple Nodes, we can:

  - use Puppet 'package' on each Node or

  - use 'exec' on the manager to ssh and yum RPMs to multiple Nodes.

  is there a better way to use Puppet for this?

Package resources are the Puppet way to do this.  Put the needed RPMs into a private Yum repository, and use Packages to ensure them installed.  If the RPMs have correct dependency information then you might not need to declare all the needed packages individually, for Yum automatically finds and installs needed dependencies when they are available.

 

- VCS comes with:

  - an Installer to install VCS software onto all Nodes;

  - an Config program to create the Cluster;

  - CLI commands to create resources to be monitored.

  so, i think that it would be best to use Exec to run these programs.

If you want Puppet to cause a program to be run on a node, then an Exec resource is the way to describe it.  Alternatively, you could consider creating a custom resource type for the purpose.  Either way, though, I am somewhat doubtful that Puppet is the best tool for this job.  What do you hope to gain over just running the Installer and Config programs directly?

John