using storeconfigs and exported resources

[Ritesh Nanda]

Hello ,

I am designing a multi-node deployment env with Puppet. Scenario i have is once one machine is created   it will have its hostname allocated dynamically , that hostname has to be used by another machine when it runs it manifest on it . 

I can get that fqdn of that machine save it in puppetdb and ask another machine to get that hostname using exported resource Is that correct?

Another scenerio in case simultaneously two users runs that module , first machine save its hostname , same time again that module is intiated by another user , now the latest machine would overwrite the entry in puppetdb.

What would be the better way of doing that.

Regards,

Ritesh

[jcbollinger]

On Wednesday, May 21, 2014 3:27:32 PM UTC-5, Ritesh Nanda wrote:

Hello ,

I am designing a multi-node deployment env with Puppet.

Ok.

 

Scenario i have is once one machine is created   it will have its hostname allocated dynamically ,

Ok.

 

that hostname has to be used by another machine when it runs it manifest on it . 

Huh?  What other machine?  What is "it"? And "it"? And "it"?

 

I can get that fqdn of that machine save it in puppetdb and ask another machine to get that hostname using exported resource Is that correct?

If you connect a client machine to your puppet master then compilation of the client's catalog can cause a resource to be exported that associates the client's hostname and fqdn.  A Host resource would fill that role quite naturally.  Supposing that you use puppetdb as your storeconfigs back end (as you should do), those data will be recorded in puppetdb.  There are multiple things you can do with the data from there, but I'm not following what specific thing you are looking to do.

The situation might be similar if you're using 'puppet apply' with local manifests instead of 'puppet agent' and a central master, but in that case there are likely to be much better ways to store local data than in puppetdb.

 

Another scenerio in case simultaneously two users runs that module , first machine save its hostname , same time again that module is intiated by another user , now the latest machine would overwrite the entry in puppetdb.

Why would you suppose that one machine's data would overwrite a different one's?  Puppet distinguishes one machine from another by their certs, unless you intentionally confuse it by manually assigning the same cert to multiple machines.  Puppetdb associates node data with node identity, so one node's data will not overwrite a different one's.


John

[Ritesh Nanda]

Hello John,

Thanks for your reply , sorry for providing less information .

I am working on developing websphere multi-node deployment module , in that i have two roles one is deploymanager another is appserver.

I have a type and provider already created for it which run on the deploymanager , which needs certain values like hostname , which would be dynamically generated when the machine is provisioned .

@@websphere_profile { $servertype:

        ensure                              => $ensure_profile,

        hostname                         => $fqdn,

        profilename                      => $ndm_profilename,

        profilepath                        => $profilepath,

        templatepath                    => $templatepath,

        cellname                         => $cellname,

        nodename                       => $nodename,

        port                                => $start_port,

        enableadminsecurity        => $enableadminsecurity,

        adminusername              => $adminusername,

        adminpassword               => $adminpassword,

        appservernodename         => $appservernodename,

        tag                                 => websphere,

}

this resource type first runs on the deploymanager machine which gets provisioned , making it exported resource will store its attributes value in puppetdb .

Once this machine is completed , a new machine would be provisioned which would be appserver , then appserver will also run this resource type providing fqdn from the puppetdb of the deploymanager  which is stored by the exported resource.

Now this whole piece would be run by a end-user who will just see a catalog , saying multi-node websphere.

Another scenerio in case simultaneously two users runs that module , first machine save its hostname , same time again that module is intiated by another user , now the latest machine would overwrite the entry in puppetdb.

Why would you suppose that one machine's data would overwrite a different one's?  Puppet distinguishes one machine from another by their certs, unless you intentionally confuse it by manually assigning the same cert to multiple machines.  Puppetdb associates node data with node identity, so one node's data will not overwrite a different one's.

Now if this catalog is ran by two users at the same time , as the exported resource will save information with the tag websphere. As there would be two multinode websphere deployment at the same time , how will puppet know how appserver gets the hostname of its correct deploymanager.

Thanks for helping me out .

Regards,

Ritesh

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/NM3vaVmOG1U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/65b2db76-2f22-4e01-abdd-56258891e2a9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

 With Regards 

 Ritesh Nanda

[jcbollinger]

On Thursday, May 22, 2014 11:17:48 AM UTC-5, Ritesh Nanda wrote:

Hello John,

Thanks for your reply , sorry for providing less information .

I am working on developing websphere multi-node deployment module

From the (little) code you provided, I infer that you are using the term "module" in a generic sense, rather than with the more specific meaning that word has in Puppet jargon.  You really should organize your code into a bona fide Puppet module.  That won't address the problem you are currently trying to solve, but it is likely to be to your advantage in a more general way.

 

, in that i have two roles one is deploymanager another is appserver.

I have a type and provider already created for it which run on the deploymanager , which needs certain values like hostname , which would be dynamically generated when the machine is provisioned .

So I think you're saying the hostname etc. are not Puppet's responsibility; you are simply dealing with the fact that you don't know them in advance.

 

@@websphere_profile { $servertype:

        ensure                              => $ensure_profile,

        hostname                         => $fqdn,

        profilename                      => $ndm_profilename,

        profilepath                        => $profilepath,

        templatepath                    => $templatepath,

        cellname                         => $cellname,

        nodename                       => $nodename,

        port                                => $start_port,

        enableadminsecurity        => $enableadminsecurity,

        adminusername              => $adminusername,

        adminpassword               => $adminpassword,

        appservernodename         => $appservernodename,

        tag                                 => websphere,

}

this resource type first runs on the deploymanager machine which gets provisioned , making it exported resource will store its attributes value in puppetdb .

"Runs on" is a somewhat confusing description, especially when you're talking about an exported resource.  Resource declarations are not executable, so they do not "run".  Most do cause a resource to be recorded in the node's catalog, to which the agent (or 'apply') will respond by taking some sort of action, but even that does not happen as a direct consequence of the declaration of an exported resource.

In part for that reason I remain uncertain whether you are running Puppet in agent / master mode or whether you are using "puppet apply" with local manifests and data.  It may make a difference.

Furthermore, I think it important that you understand that having exported resource data recorded in PuppetDB is a means to an end, not an end in itself.  The point of an exported resource is for information specific to one node to be used to configure a resource that can be applied to other nodes.

 

Once this machine is completed , a new machine would be provisioned which would be appserver , then appserver will also run this resource type providing fqdn from the puppetdb of the deploymanager  which is stored by the exported resource.

Do you mean that the deploymanager's websphere_profile resource is to be collected (imported) into the appserver's catalog, so that it will be applied to the appserver?  That would be the normal usage pattern for exported resources, but it is unclear whether that's what you mean.  It sounds more like you're saying the appserver will export its own websphere_profile resource.  But if that's so, then to what machine are these exported resources ever applied?

 

Now this whole piece would be run by a end-user who will just see a catalog , saying multi-node websphere.

That totally does not make sense.  I'm not sure exactly how you would apply a pre-compiled catalog to a random machine, but certainly that scenario won't get you additional websphere_profile resources exported, since exported resources are enrolled in the storeconfigs backend (e.g. puppetdb) during catalog compilation, not during catalog application.

 

Another scenerio in case simultaneously two users runs that module , first machine save its hostname , same time again that module is intiated by another user , now the latest machine would overwrite the entry in puppetdb.

Why would you suppose that one machine's data would overwrite a different one's?  Puppet distinguishes one machine from another by their certs, unless you intentionally confuse it by manually assigning the same cert to multiple machines.  Puppetdb associates node data with node identity, so one node's data will not overwrite a different one's.

Now if this catalog is ran by two users at the same time , as the exported resource will save information with the tag websphere. As there would be two multinode websphere deployment at the same time , how will puppet know how appserver gets the hostname of its correct deploymanager.

If you intend to be able to manage multiple distinct websphere clusters in the same Puppet context (as defined by sharing a puppetdb instance), whether concurrently or serially, then you are right that you need a means to distinguish the members of each one from the members of the others.  One way would be to assign cluster identifiers of your choosing (e.g. cluster1, cluster2, cluster20140523-rnanda-1) by which to associate members of the same cluster.  Your machines can feed that to Puppet via a custom fact.  Do note, however, that using a fact for this purpose implies that you trust the clients to correctly self-declare to which cluster they belong.

Given a cluster identifier, you could tag resources with something like "websphere-${::clusterid}" instead of with plain "websphere".


John

[Ritesh Nanda]

Hi John,

Thanks for reply again . 

That totally does not make sense.  I'm not sure exactly how you would apply a pre-compiled catalog to a random machine.

This setup would leverage vmware appdirector and  integration with puppet master , Where i will create blueprint of first  creating a vm and to apply  a particular manifest on that node. So vm creation part and puppet run on the node, node defination  is not my responsiblity . It will be done by App direct

So end user would just the catalog.

This setup you can treat it as a multinode or a cluster setup . Where i have these two components deploymanager and appserver in a puppet module. I have them both in separate classes. So that they can be called on respective nodes.

a example Now in a setup of 2 vms  there would be one deploymanager which would be created and 1 appserver (later i will work on making it user choice on no. of appserver he wants) will be created and this/these appservers would register itself with the deploymanager. 

That is the case where i want to export fqdn of deploymanager in puppetdb so that it can be used by these appserver nodes to register itself with deploymanager.

Now the custom resource type i pasted .

class websphere::deploymanager

{

websphere_profile { "Deployment manager":

        ensure                              => true,

        hostname                         =>$fqdn

}

}

 It will be  ran on the deploymanager , this resource actually makes that node as deploymanager (its a websphere command basically).

You are correct if i make this resource as exported it will not be ran on this node. As i only need fqdn of this node to be exported, is there a way to do it.

Then this custom resource is also needed to  be ran on the two  appserver nodes which would be created. Specifying the fqdn of deploymanager so that these appserver nodes can register itself with deploymanger.

class websphere::appserver 

{

websphere_profile { "Appserver1":

        ensure                              => true,

        hostname                         => (Hostname of deploymanager)

}

}

In this way these nodes will register itself with Deploymanager.

If you intend to be able to manage multiple distinct websphere clusters in the same Puppet context (as defined by sharing a puppetdb instance), whether concurrently or serially, then you are right that you need a means to distinguish the members of each one from the members of the others.  One way would be to assign cluster identifiers of your choosing (e.g. cluster1, cluster2, cluster20140523-rnanda-1) by which to associate members of the same cluster.  Your machines can feed that to Puppet via a custom fact.  Do note, however, that using a fact for this purpose implies that you trust the clients to correctly self-declare to which cluster they belong.

Given a cluster identifier, you could tag resources with something like "websphere-${::clusterid}" instead of with plain "websphere".

Yes , i thought in the same way , if i use the exported resource functionality , then if a take tag as input from the user in the form of clustername -<userid> , hope this can be used .

I am happy to give more information , thanks for helping me out.

Regards,

Ritesh Nanda 

 

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/NM3vaVmOG1U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/d79007c9-e313-4a77-829e-34cc90911700%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jcbollinger]

On Friday, May 23, 2014 3:37:33 PM UTC-5, Ritesh Nanda wrote:

Hi John,

Thanks for reply again . 

That totally does not make sense.  I'm not sure exactly how you would apply a pre-compiled catalog to a random machine.

This setup would leverage vmware appdirector and  integration with puppet master , Where i will create blueprint of first  creating a vm and to apply  a particular manifest on that node. So vm creation part and puppet run on the node, node defination  is not my responsiblity . It will be done by App direct

So end user would just the catalog.

"Pay no attention to the man behind the curtain," as the Wizard once said.  But that doesn't work for us any better than it worked for him.

I don't know details of how App direct works, but I find it highly unlikely that it involves applying pre-compiled catalogs.  It is far more likely to involve applying automatic configuration of Puppet itself, and followed be automatically running Puppet, whether in 'agent' mode or 'apply' mode, to at that time obtain and apply a catalog for the node.

This does make a difference, especially given that you are planning to use exported resources.

 

This setup you can treat it as a multinode or a cluster setup . Where i have these two components deploymanager and appserver in a puppet module. I have them both in separate classes. So that they can be called on respective nodes.

a example Now in a setup of 2 vms  there would be one deploymanager which would be created and 1 appserver (later i will work on making it user choice on no. of appserver he wants) will be created and this/these appservers would register itself with the deploymanager. 

That is the case where i want to export fqdn of deploymanager in puppetdb so that it can be used by these appserver nodes to register itself with deploymanager.

Now the custom resource type i pasted .

class websphere::deploymanager

{

websphere_profile { "Deployment manager":

        ensure                              => true,

        hostname                         =>$fqdn

}

}

 It will be  ran on the deploymanager , this resource actually makes that node as deploymanager (its a websphere command basically).

You are correct if i make this resource as exported it will not be ran on this node. As i only need fqdn of this node to be exported, is there a way to do it.

Then this custom resource is also needed to  be ran on the two  appserver nodes which would be created. Specifying the fqdn of deploymanager so that these appserver nodes can register itself with deploymanger.

class websphere::appserver 

{

websphere_profile { "Appserver1":

        ensure                              => true,

        hostname                         => (Hostname of deploymanager)

}

}

In this way these nodes will register itself with Deploymanager.

Yuck.  You might be able to get something along these lines to work, but it is not very idiomatic.  Also, it's probably a lot more work to build and maintain than you need.

In Puppet, the term "resource" refers to some reasonably self-contained aspect of target nodes that can be managed by Puppet -- a File, a service, etc..  Your "websphere_profile" seems instead to be a blob of loosely associated data with a split personality.  You are trying to put a whole (Puppet) module's worth of functionality into a single custom type.  Probably you don't actually need a custom type at all, just ordinary (Puppet) classes, built-in resource types,  and one or two defined types.

Here's a rough sketch of how I could see this working:

class websphere::deploymanager ($clusterid, ...) {
  # Exported for app servers' use:
  @@websphere::deploymanager_registration { $clusterid: }

  # ... All the Files, Packages, etc. that a DeployManager needs
}

class websphere::appserver($clusterid, ...) {
  Websphere::Deploymanager_registration<<| name == $clusterid |>>
  # ... All the Files, Packages, etc. that an app server needs
}

define websphere::deploymanager_registration (...) {
  # Whatever File, Exec, etc. is necessary for registering
  # an app server with a deploymanager.
}


With that you declare the websphere::deploymanager class on (only) the deploymanager, and you declare the websphere::appserver class on all appservers (including the deploymanager if it also functions as an app server).


John

[Ritesh Nanda]

Thanks John,

Thanks for help.

Regards,

Ritesh Nanda

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/NM3vaVmOG1U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/f228f238-c0da-49da-9420-52b224009183%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.