Re: Puppet Support for Windows

[Евгений Верещагин]

I have big problem on localized Wondows: https://groups.google.com/forum/?fromgroups=#!topic/puppet-users/HN9Fg3Q_n3M

You can't use non-ANSI characters in file\folder names, user names, etc. Now I try workaround.

воскресенье, 27 января 2013 г., 1:12:03 UTC+4 пользователь damian....@gmail.com написал:

Hi All,

I am currently looking at using PE to provide our config management (and orchestrated deployment via MCollective) for our app stack.  It is currently used to manage the Linux OS estate but not yet for Windows. I'd like to use the same tool so that the people who develop and manage apps on both OS only have a single learning curve and given PE is already used in the organisation that is my first choice.

In my initial investigation there are a number of critical functions that currently cannot be managed out the box (or via modules on PuppetForge) which i would have expected from a tool such as this. (I appreciate that Windows support on Puppet is relatively new and that I could create my own modules.  However that would mean learning Ruby *and* Puppet, diverting resource away from their main job, and convincing management to allow custom coding something that they'd expect out of the box of such a tool is going to be tricky!).

So, are there currently any plans to provide

- NTFS file support to allow detailed control of permissions settings and not relying on the very limited POSIX -> Windows mapping in the current File resource. (And yes i understand the RAL and reasons behind it, but this is kind of a deal breaker for us for the Windows side of our estate)?

- Setting the user for a Service on Windows? (I know i could probably exec out to sc.exe to achieve this but would like it config managed)

And probably not for this forum (but i know PuppetLabs employees are reading)...

- Do you have any idea of of when MCollective support in Puppet Enterprise will be provided for Windows.

Thanks,

Damian

[Josh Cooper]

Hi Damian,

On Sat, Jan 26, 2013 at 1:12 PM, <damian....@gmail.com> wrote:
> Hi All,
>
> I am currently looking at using PE to provide our config management (and
> orchestrated deployment via MCollective) for our app stack. It is currently
> used to manage the Linux OS estate but not yet for Windows. I'd like to use
> the same tool so that the people who develop and manage apps on both OS only
> have a single learning curve and given PE is already used in the
> organisation that is my first choice.
> In my initial investigation there are a number of critical functions that
> currently cannot be managed out the box (or via modules on PuppetForge)
> which i would have expected from a tool such as this. (I appreciate that
> Windows support on Puppet is relatively new and that I could create my own
> modules. However that would mean learning Ruby *and* Puppet, diverting
> resource away from their main job, and convincing management to allow custom
> coding something that they'd expect out of the box of such a tool is going
> to be tricky!).
>
> So, are there currently any plans to provide
> - NTFS file support to allow detailed control of permissions settings and
> not relying on the very limited POSIX -> Windows mapping in the current File
> resource. (And yes i understand the RAL and reasons behind it, but this is
> kind of a deal breaker for us for the Windows side of our estate)?

This is on our Windows roadmap, filed as
https://projects.puppetlabs.com/issues/13249. Recently, the priority
has increased as we've been hearing similar comments from other users.
With that said, I'm curious what use cases you're looking to solve.
Are you looking to specify the complete state of the DACL, e.g. grant
permissions to these accounts, deny to these, control inheritance? Or
a partial state, e.g. ensure administrators has full control and
ignore other ACEs that are present. Or is it a compliance issue, e.g.
ensure only administrators can write?

Also are you looking to manage DACLs on other securable objects, e.g.
registry keys.

Also are you looking to manage SACLs?

> - Setting the user for a Service on Windows? (I know i could probably exec
> out to sc.exe to achieve this but would like it config managed)

This is filed as https://projects.puppetlabs.com/issues/17706. It
would be trivial to implement, as we already have the SID resolution
code in place, and it would be similar to how we manage the user
account for scheduled tasks.

>
> And probably not for this forum (but i know PuppetLabs employees are
> reading)...
> - Do you have any idea of of when MCollective support in Puppet Enterprise
> will be provided for Windows.

I can't specify when exactly, but this is a high priority for us. The
top-level ticket is filed as
https://projects.puppetlabs.com/issues/11206. The hard work of getting
mcollective running on windows has already been done. The remaining
issues are around packaging, updating PE modules to support windows,
and better mcollective control of the puppet agent, all of which are
straightforward tasks.

Josh

--
Josh Cooper
Developer, Puppet Labs

[damian....@gmail.com]

Hi Josh,

First of all thanks for the quick reply.  

The main priorities to make Puppet usable on Windows for us would be:

1> Control complete state of the DACL for grant (we don't use deny).

2> Control inheritance on DACL (at the same time as being able to control other DACL grant entries for that object).

3> Control inheritance on SACL (we only set this at a higher level).

4> Set user account on Service.

It would also be good to have the following (although don't think it would be a showstopper for adoption):

5> Control ACL on local SMB shares.

6> Control ACL on registry.

And finally the nice to haves:

7> (Nice to have) Set DACL on parent directory but inherit permissions on all children when using source param with multiple levels of hierarchy.

8> (Nice to have) Set DACL on parent directory but inherit permissions on all children when using recurse param.

Off the top of my head (not fully worked out all our requirements with the devs yet) I don't think we control access to any other types of windows object (e.g. service)

I did start having a dig in the Puppet code for the file type and all of the building blocks are already there. I'm not sure how much effort it would be to write an ntfsfile class but I have started having a play with writing my own (in my spare time) but I've never written Ruby before so a reasonable learning curve (not least just to understand the mass of file and windows provider Puppet code let alone Ruby!). The permission setting methods are all there (e.g. set_acl and get_acl from security.rb including the protected parameter that i couldn't see a way of setting anywhere).  My plan was to replace the mode param on file.rb with a dacl param that could take some form of friendly dacl description.  The get_mode and set_mode methods could then be changed to translate between friendly dacl and real dacl rather than POSIX mode and dacl.

The friendly DACL would use something like the following to describe each ACE:

 ntfsfile { 'myfile.txt' :

    require => file,

    dacl => [ 

                  ['user1', grant, [FULL_CONTROL]],

                  ['user2', grant, [FILE_READ]],

                  ['group1', grant, [FILE_READ, FILE_WRITE, CHANGE_PERMISSIONS]],

                  ['user3', deny, [FILE_READ, FILE_WRITE, FILE_EXECUTE]]

                 ],

    inheritparent => false,

    source => 'puppet://modules/something/file.txt',

}

[Rich Siegel]

Do you have any code on github?  Perhaps we can collaborate.  I am doing a bit of windows type and provider development currently (mostly learning how ;)  I have a pendinga windows clustering provider, and a windows ad dns provider in the works.   I have also wrote a chocolatey provider that we are now officially using on 100s of servers.

[damian....@gmail.com]

no sorry, i'm only on day 2 of learning Ruby and about day 5 of Puppet.  If i'm lucky i get about an hour every other evening to look at this so progress is slow.  Once I've got something worthwhile sharing i'll post it somewhere.

My two projects are an ntfsfile type (in which you can specify a full DACL and inheritance) and a windowsservice type (which will install a service via installutil.exe if it doesn't already exist and allow you to specify a username and password).

[Josh Cooper]

Hi Damian,

On Mon, Jan 28, 2013 at 2:01 PM, <damian....@gmail.com> wrote:
> Hi Josh,
>
> First of all thanks for the quick reply.
>
> The main priorities to make Puppet usable on Windows for us would be:
>
> 1> Control complete state of the DACL for grant (we don't use deny).
> 2> Control inheritance on DACL (at the same time as being able to control
> other DACL grant entries for that object).
> 3> Control inheritance on SACL (we only set this at a higher level).

It sounds like you're wanting to model the security descriptor, and
not just the DACL.

> 4> Set user account on Service.
>
> It would also be good to have the following (although don't think it would
> be a showstopper for adoption):
> 5> Control ACL on local SMB shares.
> 6> Control ACL on registry.
>

Ideally the type should be applicable to any windows securable object,
e.g. desktops, services, etc.

> And finally the nice to haves:
> 7> (Nice to have) Set DACL on parent directory but inherit permissions on
> all children when using source param with multiple levels of hierarchy.
> 8> (Nice to have) Set DACL on parent directory but inherit permissions on
> all children when using recurse param.
>
> Off the top of my head (not fully worked out all our requirements with the
> devs yet) I don't think we control access to any other types of windows
> object (e.g. service)
>
> I did start having a dig in the Puppet code for the file type and all of the
> building blocks are already there. I'm not sure how much effort it would be
> to write an ntfsfile class but I have started having a play with writing my
> own (in my spare time) but I've never written Ruby before so a reasonable
> learning curve (not least just to understand the mass of file and windows
> provider Puppet code let alone Ruby!). The permission setting methods are
> all there (e.g. set_acl and get_acl from security.rb including the protected
> parameter that i couldn't see a way of setting anywhere).

I'd recommend using the win32-security gem[1] as a starting point and
adding whatever methods aren't yet implemented, using the puppet code
for comparison. Ideally, I'd like to see all of the ACL manipulation
done in win32-security and not in puppet.

> My plan was to
> replace the mode param on file.rb with a dacl param that could take some
> form of friendly dacl description. The get_mode and set_mode methods could
> then be changed to translate between friendly dacl and real dacl rather than
> POSIX mode and dacl.
>
> The friendly DACL would use something like the following to describe each
> ACE:
> ntfsfile { 'myfile.txt' :
> require => file,
> dacl => [
> ['user1', grant, [FULL_CONTROL]],
> ['user2', grant, [FILE_READ]],
> ['group1', grant, [FILE_READ, FILE_WRITE,
> CHANGE_PERMISSIONS]],
> ['user3', deny, [FILE_READ, FILE_WRITE, FILE_EXECUTE]]
> ],
> inheritparent => false,
> source => 'puppet://modules/something/file.txt',
> }
>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Josh

[1] https://github.com/djberg96/win32-security

[damian....@gmail.com]

Thanks Josh. Unfortunately I have had zero time to dedicate to this.

Tbh if I could just model the dacl plus control the inheritance of dacl from parent that would be enough.

I did start to write an ntfsfile class but wanted to keep all of the existing file class functionality except setting user, group, and mode so started by copying all the related ruby for that. . Thats a pretty complicated type and provider and a combination of this being my first Puppet development plus not knowing Ruby made it tough going.
The plan was to have a string property which was a JSON representation of a dacl which got munged into a combination of arrays and hashes plus an inherit property. I wrote the code to then set the dacl on the file based upon the arrays etc. This worked but was v hacky and was based upon a hard coded array (not yet written the validate and munge methods!)
Overall I'm not sure how good quality I could make this.

So..., are we likely to see any progress on the ticket for Puppetlabs to release an official version for this?
Would it help speed things up if a request came from a PE customer (with 1500 node licence)?

[damian....@gmail.com]

Hi Josh,

Did PuppetLabs ever get anywhere with ticket 13249.  I'm guessing not as it hasn't been updated for months.  I see that PE 3 is now available and that there has been plenty of activity on the Windows side of things (we are about to start a trial with it on some of our Windows estate as Windows support especially MCollective is much better).
Do you have any sort of timescale for this ticket.  I'd like to vote on it but the "gatekeepers" of our PE login details are not letting me near it (the sysadmins own it, we're an app development department and having some fun and games with them trying to do continuous deployment type stuff!).

If you have any early release type versions of this I might be able to help out with some testing etc.

Damian

>> Josh quote >>

This is on our Windows roadmap, filed as
https://projects.puppetlabs.com/issues/13249. Recently, the priority
has increased as we've been hearing similar comments from other users.
With that said, I'm curious what use cases you're looking to solve.
Are you looking to specify the complete state of the DACL, e.g. grant
permissions to these accounts, deny to these, control inheritance? Or
a partial state, e.g. ensure administrators has full control and
ignore other ACEs that are present. Or is it a compliance issue, e.g.
ensure only administrators can write?
>>

[Josh Cooper]

Hi Damian,

On Sun, Jul 28, 2013 at 1:03 PM, <damian....@gmail.com> wrote:

Hi Josh,

Did PuppetLabs ever get anywhere with ticket 13249.  I'm guessing not as it hasn't been updated for months.  I see that PE 3 is now available and that there has been plenty of activity on the Windows side of things (we are about to start a trial with it on some of our Windows estate as Windows support especially MCollective is much better).

Yes, mcollective is fully supported on windows now!

 

Do you have any sort of timescale for this ticket.  I'd like to vote on it but the "gatekeepers" of our PE login details are not letting me near it (the sysadmins own it, we're an app development department and having some fun and games with them trying to do continuous deployment type stuff!).

I don't have a time estimate, other than to say that we are working on a set of improvements to file system management, including NTFS ACLs, symlinks, and some bug fixes, and it's one of our top priorities, along with powershell and reboot support.

If you have any early release type versions of this I might be able to help out with some testing etc.

That'd be great! Managing the ACL via Win32 API's is not hard (we do much of it today). The hard part is deciding how best to model the ACL in puppet. Should it be a separate resource type or a property of the file type? Do we create a new file acl type with multiple providers, e.g. windows, solaris, etc? If so, what parameters and properties are common, and which are unique to a specific provider, e.g. protected on windows?

Josh

-- 

Josh Cooper

Developer, Puppet Labs

Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
Register now and take advantage of the Final Countdown discount - save 15%!

[cha...@lyricalsoftware.com]

Hi Josh,

On Sun, Jul 28, 2013 at 1:03 PM, <damian....@gmail.com> wrote:

<snip>

I don't have a time estimate, other than to say that we are working on a set of improvements to file system management, including NTFS ACLs, symlinks, and some bug fixes, and it's one of our top priorities, along with powershell and reboot support.

That's amazing news. Powershell support alone would be killer - I had to wrap nearly my entire universe with cmd.exe to make like easier - to the point of writing a shared module for this pain. Powershell can't come fast enough :)

 

<snip> 

That'd be great! Managing the ACL via Win32 API's is not hard (we do much of it today). The hard part is deciding how best to model the ACL in puppet. Should it be a separate resource type or a property of the file type? Do we create a new file acl type with multiple providers, e.g. windows, solaris, etc? If so, what parameters and properties are common, and which are unique to a specific provider, e.g. protected on windows?

Hopefully my $0.02 can we worth something here ;) I'd argue that it's really a separate resource type - since the ACL is related to the user space. If you're going to extend it to multiple providers (solaris as per your example) it's really similar in idea to RBAC. In fact, if you look at Windows ACLs, RBAC, and set/get facl you pretty much have a new type.  Or at least that's what I'd hope :)

 

Josh

-- 

Josh Cooper

Developer, Puppet Labs

Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
Register now and take advantage of the Final Countdown discount - save 15%!

--c

Chayim Kirshen

Founder, Lyrical Software

@lyricaldevops | http://www.lyricaldevops.com

[Josh Cooper]

On Wed, Jul 31, 2013 at 6:22 AM, <cha...@lyricalsoftware.com> wrote:

Hi Josh,

On Sun, Jul 28, 2013 at 1:03 PM, <damian....@gmail.com> wrote:

<snip>

I don't have a time estimate, other than to say that we are working on a set of improvements to file system management, including NTFS ACLs, symlinks, and some bug fixes, and it's one of our top priorities, along with powershell and reboot support.

That's amazing news. Powershell support alone would be killer - I had to wrap nearly my entire universe with cmd.exe to make like easier - to the point of writing a shared module for this pain. Powershell can't come fast enough :)

I have a powershell provider here: http://forge.puppetlabs.com/joshcooper/powershell. And as of puppet 3.2, that the module tool supports windows, so you can do:

C:\>puppet module install joshcooper-powershell

C:\>puppet apply -e "exec { 'Write-Host hello': provider=> powershell, logoutput => true }"

Notice: /Stage[main]//Exec[Write-Host hello]/returns: hello

Notice: /Stage[main]//Exec[Write-Host hello]/returns: executed successfully

We've recently made some improvements around powershell invocation. See https://github.com/joshcooper/puppetlabs-powershell/issues for more info.

Josh

[jcbollinger]

On Wednesday, July 31, 2013 8:22:01 AM UTC-5, cha...@lyricalsoftware.com wrote:

Hopefully my $0.02 can we worth something here ;) I'd argue that it's really a separate resource type - since the ACL is related to the user space. If you're going to extend it to multiple providers (solaris as per your example) it's really similar in idea to RBAC. In fact, if you look at Windows ACLs, RBAC, and set/get facl you pretty much have a new type.  Or at least that's what I'd hope :)

And of course some Solaris is by no means the only Unix-y OS with ACL support.  It is available on Linux, too, at least for the most frequently used filesystems, and I'm sure there are others.  I'm inclined to agree that a type aimed at broad ACL / RBAC support would be a win.


John

[Chayim Kirshen]

On Aug 1, 2013 12:56 AM, "Josh Cooper" <jo...@puppetlabs.com> wrote:
>
>
>
>
> On Wed, Jul 31, 2013 at 6:22 AM, <cha...@lyricalsoftware.com> wrote:
>>
>> Hi Josh,
>>
>>> On Sun, Jul 28, 2013 at 1:03 PM, <damian....@gmail.com> wrote:
>>>>
>>>> <snip>
>>>
>>> I don't have a time estimate, other than to say that we are working on a set of improvements to file system management, including NTFS ACLs, symlinks, and some bug fixes, and it's one of our top priorities, along with powershell and reboot support.
>>
>>
>> That's amazing news. Powershell support alone would be killer - I had to wrap nearly my entire universe with cmd.exe to make like easier - to the point of writing a shared module for this pain. Powershell can't come fast enough :)
>
>
> I have a powershell provider here: http://forge.puppetlabs.com/joshcooper/powershell. And as of puppet 3.2, that the module tool supports windows, so you can do:
>
> C:\>puppet module install joshcooper-powershell
> C:\>puppet apply -e "exec { 'Write-Host hello': provider=> powershell, logoutput => true }"
> Notice: /Stage[main]//Exec[Write-Host hello]/returns: hello
> Notice: /Stage[main]//Exec[Write-Host hello]/returns: executed successfully
>
> We've recently made some improvements around powershell invocation. See https://github.com/joshcooper/puppetlabs-powershell/issues for more info.
>
> Josh
>
> --
> Josh Cooper
> Developer, Puppet Labs

Fantastic thank you! I'm going to bang on this next week.

Cheers,
--c

Chayim Kirshen
Founder, Lyrical Software
@lyricaldevops

>

> Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
> Register now and take advantage of the Final Countdown discount - save 15%!
>

> --
> You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/yKZAWODowGA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

> To post to this group, send email to puppet...@googlegroups.com.

> Visit this group at http://groups.google.com/group/puppet-users.

[Josh Cooper]

Hi John,

Yep, I agree. Now, how exactly to map the type across different implementations?

Windows ACLs support inheritance. An ACL can be marked as protected, breaking inheritance, and for directories, everything below it.

ACEs specify a subject (SID) and the rights that are granted/denied. This is a bitfield, though users are more typically used to saying "Full Control" or "Read & Execute".

Windows ACEs can either be allow or deny, the order matters, and if no ACEs match, access is denied.

An ACE for a directory can be marked as object-inherit and/or container-inherit. This doesn't affect the effective permissions on the directory, only files and subdirectories, respectively.

How are these similar & different to Unix-y ACLs?

Josh

 

[jcbollinger]

On Thursday, August 1, 2013 5:45:29 PM UTC-5, Josh Cooper wrote:

Hi John,

On Thu, Aug 1, 2013 at 6:00 AM, jcbollinger <John.Bo...@stjude.org> wrote:

On Wednesday, July 31, 2013 8:22:01 AM UTC-5, cha...@lyricalsoftware.com wrote:

Hopefully my $0.02 can we worth something here ;) I'd argue that it's really a separate resource type - since the ACL is related to the user space. If you're going to extend it to multiple providers (solaris as per your example) it's really similar in idea to RBAC. In fact, if you look at Windows ACLs, RBAC, and set/get facl you pretty much have a new type.  Or at least that's what I'd hope :)

And of course some Solaris is by no means the only Unix-y OS with ACL support.  It is available on Linux, too, at least for the most frequently used filesystems, and I'm sure there are others.  I'm inclined to agree that a type aimed at broad ACL / RBAC support would be a win.

Yep, I agree. Now, how exactly to map the type across different implementations?

Windows ACLs support inheritance. An ACL can be marked as protected, breaking inheritance, and for directories, everything below it.

ACEs specify a subject (SID) and the rights that are granted/denied. This is a bitfield, though users are more typically used to saying "Full Control" or "Read & Execute".

Windows ACEs can either be allow or deny, the order matters, and if no ACEs match, access is denied.

An ACE for a directory can be marked as object-inherit and/or container-inherit. This doesn't affect the effective permissions on the directory, only files and subdirectories, respectively.

How are these similar & different to Unix-y ACLs?

Please allow me to refine my terminology from "Unix-y" to "POSIX".  Here's a document that does a pretty good job of explaining POSIX ACLs: http://users.suse.com/~agruen/acl/linux-acls/online/.

To answer your questions more directly, however:


ACL Inheritance:

POSIX defines "default" ACLs for directories, which provide the closest analog to Windows ACL inheritance.  A directory's default ACL is assigned as the ACL of each file or directory created therein, and also as the default ACL of each directory created therein (subject to further restriction according to the requested initial mode for the file/directory).  POSIX does not differentiate between files and directories in this regard, except inasmuch as only directories have default ACLs.

ACLs are bound directly to each file and directory; they do not automatically change if their parent directory's default ACLs are changed, and access control decisions are based only on Files own ACLs (and I suspect this is true under the covers for Windows, too).  POSIX differs from Windows in not defining features for automatically or implicitly updating the ACLs of a directory's contents when that directory's default ACL is modified: POSIX default ACLs are relevant only at the creation of new files and subdirectories.


ACL Scope and Structure:

POSIX ACEs reflect and extend the standard POSIX file permission scheme, allowing for read, write, and execute permission to be granted (or not) to specified users or groups.  The traditional POSIX 'group' permissions map to a mask of the maximum permissions that any ACE other than the owner's or 'other' can grant.

Access attempts that are not otherwise mapped to an ACE use the 'other' ACE that all files have; this is analogous to Windows's "Everyone".  It does not necessarily grant any access.

There is no affirmative permission denial as such, only absence of permission grant.  It amounts to the same thing for users because if there is an ACE matching the UID of the process requesting access then that ACE determines access, or lack thereof.  For groups, however, access can be granted through any of the process's groups, even if others of its groups do not have the requested access.

POSIX ACL order is not significant, but ACE specificity is.  When a user-specific ACE is applicable, it determines access, possibly in conjunction with the mask ACE.  Otherwise, when one or more group-specific ACEs are applicable, they jointly determine access, together with any mask ACE.  Only if no other ACE applies is the 'other' ACE relevant.


John