Recursively chown'ing a directory

[Matt Coddington]

Hi All,
Prior to Ansible 0.7, the file module would recursively chown
directories, now it doesn't (seems like the right behavior; i was
surprised when it did recurse, but that's actually what i wanted in
this case :)). Is there another way I should be doing this besides
figuring out each directory I want and maybe using with_items?

thanks!
matt

[Michael DeHaan]

Yeah, we fixed the glitch.

I'd accept a patch for a recurse option that ads the '-R' ... I almost
thought we already had that but it looks like no.

The hard part is computing 'changed' in that instance, to which point
I'd probably say returning 'True' all the time is plenty sufficient
when recurse is set.

[Matt Coddington]

On Mon, Sep 17, 2012 at 7:22 PM, Michael DeHaan
<michael...@gmail.com> wrote:
> Yeah, we fixed the glitch.
>
> I'd accept a patch for a recurse option that ads the '-R' ... I almost
> thought we already had that but it looks like no.
>
> The hard part is computing 'changed' in that instance, to which point
> I'd probably say returning 'True' all the time is plenty sufficient
> when recurse is set.

I've got a commit that attempts this at
https://github.com/mcodd/ansible/commit/2ec3db0643d52859f45c6c8a846fc8864c77e956.
Apparently Python doesn't have built-in recursion for chmod/chown so
you have to use os.walk, but when you do that you actually get
computing "changed" for free because you can just use the existing
functions in the file module while looping over the files and
directories (I'm a pretty novice Python programmer, but this bears
itself out based on a few basic tests I've run since that commit in my
local environment).

Didn't submit a pull request yet because I'm on the fence about
whether and/or how to recurse for chmod'ing. Per the commit above, it
does just what you tell it to do; e.g. if I say recurse=yes and tell
it mode=0644 then all my files and directories will become 0644. Even
thought that's what I asked for, I probably didn't mean it (I probably
meant I wanted all the files to be 644 but that I still wanted the
execute bit set on my directories). os.walk makes it easy to just
chmod the files instead of the directories, but that's also probably
not what everyone wants (and certainly not what everyone would
expect). Should we just not allow recurse for mode, leave it as-is,
or... ?

thanks,
matt

[Michael DeHaan]

Hmm, I'd be tempted to just add the -R, as the directory tree could be
/quite/ large, and coreutils is optimized for the task.

I would guess it would want to do files /and/ directories, and the
recurse would also affect chgrp.

I'd say we could just leave it out though, as executing via shell
isn't terrible.

[Matt Coddington]

Are you suggesting that we just use os.system and the OS's chmod/chown
when recurse is set, to avoid potential scaling problems with using
Python when the directory trees are huge? If so, I agree w/you that
just executing via shell is a better alternative, and putting
os.system calls in the file module just for this purpose would just
clutter it without adding much value. That said, it was kind of neat
to realize we got the appropriate value for Changed so easily by doing
this with os.walk... I think there's some value there, but don't know
enough about Python to understand the scaling limitations or maybe
other reasons it's not such a great idea.

matt

On Mon, Sep 17, 2012 at 9:37 PM, Michael DeHaan

[Michael DeHaan]

Yes, basically I'm thinking of what happens when someone decides to
try it on a large directory tree.

[Aleksej Romanov]

I tried.

$ cp -r /usr/ /tmp/ &>/dev/null
$ find /tmp/usr | wc -l
130615
$ time chgrp -R user /tmp/usr/

real 0m0.372s
user 0m0.048s
sys 0m0.320s
$ time ansible -c local -i /tmp/i 127.0.0.1 -m file -a 'path=/tmp/usr \
group=user state=directory recurse=yes' &>/dev/null

real 0m7.696s
user 0m5.488s
sys 0m2.192s

So it is much slower, but still only 7 seconds with quite large directory.
Assuming that one usually would use this for own project's files, not /,
it should be quick enough.

According permissions for files and dirs, may be it is reasonable to
have separate parameters for each one, not 'mode' for both, so it will
be 'permissions_mode_for_files=644 permissions_mode_for_directories=755'

[senorsmile]

I could definitely use this: 

permissions_mode_for_files=644 permissions_mode_for_directories=755

or something like it.  Right now I have to run manual shell with find like this: 

find /some/dir/ -type d -exec /bin/chmod 755 {} \;

find /some/dir/ -type f -exec /bin/chmod 644 {} \;

[Ben Hoyt]

Yeah, I'm seeing this same thing -- trying to change the owner of a large (existing) directory recursively is taking 10-20 times as long using the "file" module vs just using chown -R, so I've opted to use the latter in our playbooks.

[Michael DeHaan]

FYI:  you are replying to a mailing list post from last October.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/300a6f17-ffe8-40bf-b505-020a9fc0ffd1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Hoyt]

Understood -- the intention was to bump this up as it's still an issue with Ansible 1.5.5. -Ben

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/4PEgOLWfLeo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgz0WghtT9zwrCqKg8w1vCreUT-%2BEjy%3DQO0U_5Lq0K1oSg%40mail.gmail.com.

[Michael DeHaan]

See if there's a ticket filed and file one would be better, we really don't bump old threads on this list.  

I also don't consider this an issue so much more as a feature request / enhancement idea, where a pull request that makes it happen would probably be ideal.

You can also keep using the chown.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAL9jXCHzo3TPdAd1os24GpD9OUSgENPkK1Gumyps10SkiOkUog%40mail.gmail.com.