Values with leading zeroes (file mode)

[Giovanni Tirloni]

Hi,

When specifying a variable for file/directory mode, if there is a zero
in front of the numbers (e.g. 0775) then it gets treated as an octal.

The workaround is to quote the number, then the file module will
correctly pass it to chmod (i.e. setuid bit is not set).

It seems it hits an exception in set_mode_if_different()
(lib/ansible/module_utils/basic.py) but I'm not sure it should be
treated there. Does the number gets interpreted as octal by Jinja2 or
Ansible? Please let me know if it's worth filing an issue about this.

Regards,
Giovanni

[Josef Špak]

Hi Giovanni,

personally I'd say it's worth filing an issue (unless someone else already did). I've run into this a couple of times myself and it's not pleasant to mess up file modes by accident. 

This is one of the few cases when writing ansible tasks doesn't "just work" but has quirks you have to work around. Would be great to get rid of this one.

Josef

[Matt Martz]

The octal format of a leading 0 comes from python itself.  I wrote up an explanation in an issue a while back which you can see at https://github.com/ansible/ansible/issues/9196#issuecomment-57168074

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Giovanni Tirloni]

On Tue, 03 Feb 2015 09:52 -0600, Matt Martz <ma...@sivel.net> wrote:
> The octal format of a leading 0 comes from python itself. I wrote up an
> explanation in an issue a while back which you can see at
> https://github.com/ansible/ansible/issues/9196#issuecomment-57168074

It seems in the case of the file module, it'd be desirable to have the
value treated as chmod interprets it so there is no confusion.

I'm thinking some additional processing in the module itself would be
require. Opinions? I'm up for submiting a PR if we agree something here.

As it's today, we have places where we need quotes because of Jinja,
other places we don't, one place things are treated as integers, in
others as octal, sometimes it's best to use Ansible's concise syntax,
other times it's better to go YAML. I think the philosophical question
here is how much Python and Jinja2 we want users to know before using
Python? :) </rant level="mild">

Giovanni

[Toshio Kuratomi]

Couple things: I've tested this a few times and I just want to be
clear that you're seeing what I am:

Octal works fine: ie:

file:
path: /var/tmp/test
mode: 0644

Strings also work fine:
file:
path: /var/tmp/test
mode: "644"

Decimal integers that look like the octal modes do not work fine:
file:
path: /var/tmp/test
mode: 644

Correct?

Now my thoughts: I don't think we can do more processing without
opening up different cornercases. Both the octal specification and
the decimal specification are numbers. That means that by the time
the module gets a hold of the value it no longer knows whether the
user specified the number as base 10 or base 8. It's simply an int.
This makes any preprocessing we'd do ambiguous. For instance, let's
say the file module gets a mode parameter of 420. If we're going to
try to second guess the user we end up having to ask the question is
that -r---w---- (user specified 420 and meant 0420) or is it
-rw-r--r-- (user specified 0644 and meant 0644) with no way to tell
for sure which is which.

If we didn't care about backwards compatibility we could tell ansible
that the value of mode must be a string. Then we could make sure that
we parsed the string as an octal value always. However, that seems
like a somewhat high price to pay here since most greybeards
understand that this is an octal value and needs to be prepended with
a zero in most programming contexts. Middle ground might be to
clarify the documentation for the mode parameter. That seems to be
the route that languages like php take:
http://php.net/manual/en/function.chmod.php

-Toshio

[Giovanni Tirloni]

On Tue, 03 Feb 2015 23:07 -0800, Toshio Kuratomi <tkur...@ansible.com>
wrote:

> Couple things: I've tested this a few times and I just want to be
> clear that you're seeing what I am:
>
> Octal works fine: ie:
>
> file:
> path: /var/tmp/test
> mode: 0644
>
> Strings also work fine:
> file:
> path: /var/tmp/test
> mode: "644"
>
> Decimal integers that look like the octal modes do not work fine:
> file:
> path: /var/tmp/test
> mode: 644

I faced the issue with the following organization:

# roles/common/vars/main.yml

my_dirs:
- path: /path/to/dir
owner: user
group: group
mode: 0775

# roles/common/tasks/main.yml

- name: "Create my dirs"
file:
path: "{{ item.path }}"
owner: "{{ item.owner }}"
group: "{{ item.group }}"
mode: "{{ item.mode }}"
state: directory
with_items: my_dirs

The workaround was to specify the mode between quotes in vars/main.yml.

Thinking now, the {{ }} might be unnecessary is this specific case? I
think I naturally defaulted to quote everything in in my tasks but might
not be the right thing to do here.

Is it right to assume the playbooks are processed in the following way?

[text files] -> [jinja] -> [ansible runner]

> Now my thoughts: I don't think we can do more processing without
> opening up different cornercases. Both the octal specification and
> the decimal specification are numbers. That means that by the time
> the module gets a hold of the value it no longer knows whether the
> user specified the number as base 10 or base 8. It's simply an int.
> This makes any preprocessing we'd do ambiguous. For instance, let's
> say the file module gets a mode parameter of 420. If we're going to
> try to second guess the user we end up having to ask the question is
> that -r---w---- (user specified 420 and meant 0420) or is it
> -rw-r--r-- (user specified 0644 and meant 0644) with no way to tell
> for sure which is which.

Yeah, this makes sense.

> If we didn't care about backwards compatibility we could tell ansible
> that the value of mode must be a string. Then we could make sure that
> we parsed the string as an octal value always. However, that seems
> like a somewhat high price to pay here since most greybeards
> understand that this is an octal value and needs to be prepended with
> a zero in most programming contexts. Middle ground might be to
> clarify the documentation for the mode parameter. That seems to be
> the route that languages like php take:
> http://php.net/manual/en/function.chmod.php

It looks like a good compromise. Are the docs generated from the
docstring in the modules or somewhere else?

Thanks,
Giovanni

[Toshio Kuratomi]

On Wed, Feb 4, 2015 at 2:10 AM, Giovanni Tirloni <g...@gtirloni.com> wrote:
> On Tue, 03 Feb 2015 23:07 -0800, Toshio Kuratomi <tkur...@ansible.com>
> wrote:
> I faced the issue with the following organization:
>
> # roles/common/vars/main.yml
>
> my_dirs:
> - path: /path/to/dir
> owner: user
> group: group
> mode: 0775
>
> # roles/common/tasks/main.yml
>
> - name: "Create my dirs"
> file:
> path: "{{ item.path }}"
> owner: "{{ item.owner }}"
> group: "{{ item.group }}"
> mode: "{{ item.mode }}"
> state: directory
> with_items: my_dirs
>
> The workaround was to specify the mode between quotes in vars/main.yml.
>
> Thinking now, the {{ }} might be unnecessary is this specific case? I
> think I naturally defaulted to quote everything in in my tasks but might
> not be the right thing to do here.
>

Ahhh.... so here the variable is first defined as a number in a
variable. Then when we want to make use of the variable it becomes a
string because we have to quote it to get jinja2 to process it. That
is unintuitive :-(

There's two ways that I can see to work around that in the playbook --
as you've done, make sure that the value is a string from start to
finish. The other way is to make sure that it is a number from start
to finish by using the jinja2 int filter like this:

file:
mode: "{{ item.mode|int }}"

I think using strings throughout is the easier workaround for people
to understand but both should work.

> Is it right to assume the playbooks are processed in the following way?
>
> [text files] -> [jinja] -> [ansible runner]
>

There's also the yaml parserl in there like this:

[text file] -> PyYAML -> jinja2 -> ansible runner

In my example where vars weren't involved it was pyyaml that was
interpreting the value as a number rather than a string.

>> If we didn't care about backwards compatibility we could tell ansible
>> that the value of mode must be a string. Then we could make sure that
>> we parsed the string as an octal value always. However, that seems
>> like a somewhat high price to pay here since most greybeards
>> understand that this is an octal value and needs to be prepended with
>> a zero in most programming contexts. Middle ground might be to
>> clarify the documentation for the mode parameter. That seems to be
>> the route that languages like php take:
>> http://php.net/manual/en/function.chmod.php
>
> It looks like a good compromise. Are the docs generated from the
> docstring in the modules or somewhere else?
>

By and large. But file mode parameters for file are documented
specially because they're shared among several different modules.
You'll find them here:
https://github.com/ansible/ansible/blob/devel/lib/ansible/utils/module_docs_fragments/files.py

-Toshio