How to get an input file to a facter ?

[eduardo]

Dears all,

I was testing my localusers facter by puppetmaster fileserver but i'd
got in error

Could not retrieve localusers: No such file or directory - /etc/
puppet/whitelist

I was pretending the file was served by fileserver of puppetmaster
doing in init.pp :

file { "/etc/puppet/whitelist":
ensure => present,

Just before to call a facter.


I don't pretty sure but seems to me a issue about workflow

Client pluginsync -> Client discover system Facts -> Master
compilation -> Client apply catalog -> Client report.


Is there any way to get a file from puppetmaster to be read it by a
facter ?.

If it's not, I appreciate any suggestion about it.

Thanks in advanced,
eduardo.

[eduardo]

To be more clear about my first intend. it had init.pp like :

file { "/etc/puppet/whitelist":
ensure => present,

source => 'puppet:///files/whitelist',
}

$users_local = split($localusers, '[,]')



----- facter
require 'etc'

Facter.add("localusers") do
setcode do

# Whitelist users to exclude for checking valid ssh users

whitelist = Array.new

File.readlines("/etc/puppet/whitelist").each { |line|

whitelist << line.chomp
}
-----

[jcbollinger]

 

Facter runs after pluginsync (if enabled) and before any resources (such as your whitelist file) are synchronized.  It must be so because the master needs the node facts to compile a catalog, and the agent uses the catalog to synchronize resources.

What you are attempting to do sounds dubious, however.  If the master knows what users are supposed to be whitelisted (in order to provide the needed file) then it shouldn't need facter to tell it.


John

[eduardo]

Thanks you john for your answer. I comment you something that work
well for me.

I think get a solution while reading puppet cookbook. It's based on
run stages.

I have site.pp :

import 'sync_files.pp'

Then, sync_files.pp is :

class sync_files {
notify { "sync whitelist file": }

file { "/etc/puppet/whitelist":
ensure => present,

owner => root,
group => root,
mode => 644,

source => 'puppet:///files/whitelist',
}
}

And finally insert the following two sentences into class updssh

stage { "first": before => Stage["main"] }

class { "sync_files": stage => "first" }

That's all. Testing results are good enough.

Regards,
eduardo.

[eduardo]

John the whitelist is a dynamic file create/update by administrators,
so puppetmaster don't know about whilelist's content.
I pretending to get advantage of fileserver funcionality (instead of
any other remote copy tool like rsync) in order to get centralized
copy of the file whitelist to all nodes.

Regards,
eduardo.

[eduardo]

I need a facter because each node have different users. The facter
return a list of local users that not are into whitelist (unique).

----------------------------

whitelist = Array.new
File.readlines("/etc/puppet/whitelist").each { |line|
whitelist << line.chomp
}

locals = Array.new

Etc.passwd {|u|
locals << u.name unless u.dir[0,5] != "/
home"
}

ret = locals - whitelist
ret.join(',')

----------------------------

[jcbollinger]

On Thursday, July 5, 2012 11:16:56 AM UTC-5, eduardo wrote:

John the whitelist is a dynamic file create/update by administrators,
so puppetmaster don't know about whilelist's content.

If the file is served by the master, then the master can know whatever it needs to know about its content.  Ideally, Puppet would have the data in raw form, so as to generate the file for nodes, but it can also be made to parse the file it's about to serve by any one of several means.  For example, you can call a parsing script (or maybe just 'cat') via the generate() function.  Or if the file is generated via a template then you can store the generated content in a class variable, or else just process the template again.  Or you can write a class in Ruby DSL that does pretty much anything you want.
 

 I pretending to get advantage of fileserver funcionality (instead of
any other remote copy tool like rsync) in order to get centralized
copy of the file whitelist to all nodes.

Serving the file via Puppet is a perfectly fine and appropriate thing to do.  The point, however, is that the file doesn't need to have already been served for the master (who is serving it) to know what's in it.  Facter is the wrong way to get at the data in that case.


John

[jcbollinger]

On Thursday, July 5, 2012 10:58:54 AM UTC-5, eduardo wrote:

Thanks you john for your answer. I comment you something that work
well for me.

 I think get a solution while reading puppet cookbook. It's based on
run stages.

 I have site.pp :

 import 'sync_files.pp'

 Then, sync_files.pp is :

 class sync_files {
  notify { "sync whitelist file": }
  file { "/etc/puppet/whitelist":
         ensure => present,
         owner => root,
         group => root,
         mode => 644,
         source => 'puppet:///files/whitelist',
      }
}

 And finally insert the following two sentences into class updssh

     stage { "first": before => Stage["main"] }

     class { "sync_files": stage => "first" }

 That's all. Testing results are good enough.

 
What you have posted does not solve the problem you originally posed.  Specifically, it does not cause the whitelist to be updated prior to being processed by facter.

Fact collection happens once each cycle, after plugin sync, and before any compilation.  A single catalog is then produced and applied, containing all the resources declared in all the stages.  Run stages merely affect the order in which resources are applied relative to one another; they cannot make any resources be applied before fact collection.

I can only suppose that you are picking up a stale whitelist in your tests.


John

[jcbollinger]

On Thursday, July 5, 2012 11:33:03 AM UTC-5, eduardo wrote:

I need a facter because each node have different users. The facter
return a list of local users that not are into whitelist (unique).

Maybe it would be fruitful to come at this from a different direction.  For what purpose do you want this data?  There may be a better way to achieve your objective.


John

[eduardo]

Hi john, This data are need for check a valid users on nodes. We are
pretending massive load accounts by ENC. The batch (json) is prepare
by external program which, in our scenario is a normal way to create
accounts. But users can create new accounts by 'hand' when they log in
because they have sudo, for those new ones we want to set disable
(nologin). Meanwhile there are a set of user admins (we are) which
should be not disable even when they are not into batch json.
They are into whitelist (admin,deploy,etc,etc) , so they should be
exclude at checkout for disable accounts. That is done by the facter.

As you can see it's non-normal scenario.

Do you see any problem about solution based on run stages ?.

Thanks you,
eduardo.

[jcbollinger]

On Friday, July 6, 2012 1:05:47 PM UTC-5, eduardo wrote:

Hi john, This data are need for check a valid users on nodes. We are
pretending massive load  accounts by ENC. The batch (json) is prepare
by external program which, in our scenario is a normal way to create
accounts. But users can create new accounts by 'hand' when they log in
because they have sudo, for those new ones we want to set disable
(nologin). Meanwhile there are a set of user admins (we are) which
should be not disable even when they are not into batch json.
 They are into whitelist (admin,deploy,etc,etc) , so they should be
exclude at checkout for disable accounts. That is done by the facter.

 As you can see it's non-normal scenario.

Yes.  Here's what I think you're saying:

1) Your ENC is going to feed data for a large number of users to the puppetmaster, either via a class parameter or via a global variable.

2) Your custom fact is going to report to the puppemaster some or all of the known system users, excluding all users on the whitelist

3) Some class is going to combine the data from these sources to generate User resources that manage the ENC-specified and discovered users to, among other things, disable login for the users that do not appear in the first source.

I still don't see the point of relying on a client-side whitelist, though.  Why do you need to filter whitelisted users from the fact value on the client side?  Why can't you do it on the master instead?

  Do you see any problem about solution based on run stages ?.

As I said before, run stages cannot overcome the problem of ensuring the whitelist is up to date on the client when facter runs.  Unless you can tolerate use of an outdated file, an external client-side whitelist simply will not work.

If you are using pluginsync (recommended) then possibly you could cook the whitelist entries directly into the custom fact code.  That's ugly, but it's your best bet for ensuring that the custom fact uses an up-to-date whitelist.

Overall, though, I think your plan runs very much against the Puppet grain.  I have been known sometimes to admonish folks that "Puppet is not a script engine," but never before have I heard a deployment plan that tried so hard to use it as one.  If you continue this way then I don't think you'll be satisfied with the result.  It may be worthwhile to consider other configuration management systems.

From a higher perspective, if you're fighting node admins who have sufficient privilege to manage users then you have chosen a losing game.  If you give me that much access to a box then it's mine.  Do not give administrative privileges to people you do not trust.


John

[eduardo]

John I really appreciate all your effort to help me.
You are very close to my scenario (points 1 , 2 , 3)

> I still don't see the point of relying on a client-side whitelist, though.
> Why do you need to filter whitelisted users from the fact value on the
> client side? Why can't you do it on the master instead?

Ok, i'm going to re-check it.

> If you are using pluginsync (recommended) then possibly you could cook the
> whitelist entries directly into the custom fact code. That's ugly, but
> it's your best bet for ensuring that the custom fact uses an up-to-date
> whitelist.

Well this way i'll have to change a facter whenever whitelist change.
The up-to-date is not critical between cycle agent (30 min) in this
case.

> Overall, though, I think your plan runs very much against the Puppet
> grain. I have been known sometimes to admonish folks that "Puppet is not a
> script engine," but never before have I heard a deployment plan that tried
> so hard to use it as one. If you continue this way then I don't think
> you'll be satisfied with the result. It may be worthwhile to consider
> other configuration management systems.

Agree, i'm taking note from you and from book 'Pulling Strings with
Puppet' when say :

Caution ➡ Puppet is probably not ideal to populate large numbers of
users and groups to provide user access for nodes and applications.
Puppet is best used to populate nodes with users for running
applications and services, systems administration, and management.

> From a higher perspective, if you're fighting node admins who have
> sufficient privilege to manage users then you have chosen a losing game.

> If you give *me* that much access to a box then it's mine. Do not give

> administrative privileges to people you do not trust.

Agree too John , but i have not choice , it's an scenario created by
my employers.

Thanks you. I'm going to recheck filter whitelisted.

Best Regards,
eduardo.

> If you give *me* that much access to a box then it's mine.  Do not give

[eduardo]

Hi John, I build a custom function to filter whitelisted.
I didn't realized that custom functions can call facter by lookupvar.

Thanks you for your suggestions.

Best Regards,
eduardo.