Ansible 1.5 is released!

[Michael DeHaan]

Or should we say, it escapes!

And it's on PyPi now.

http://blog.ansibleworks.com/2014/02/28/ansible-1-5-released/

Read the blog link above for details, and if you don't want to download via pip you can hit releases.ansible.com too.

--Michael

[Nolan Darilek]

Awesome, I've been looking forward to the new Docker modules in this release.

Question, I've uninstalled Fedora's Ansible package so I can use the latest from Pypi. I run:

pip install --user ansible

But the package still tries to create files outside my home directory. Is there a way around this?

I get that I'll have to set my own module search path, but I'd like it to respect my request to install with --user. Thanks.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEVJ8QMv8D12U0eyVqoqDwkXDqjUKtdOe5Skz%3DPzZOqcdanydA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
!DSPAM:5310f819215451461565874!

[Walid]

I was doing brew update, upgrade, and noticed that Ansible is 1.5 before reading it over here. thanks to everyone that made it possible. 

[Dustin C. Hatch]

On 02/28/2014 02:56 PM, Michael DeHaan wrote:

> via pip you can hit releases.ansible.com <http://releases.ansible.com> too.
>
> --Michael
>

Great work, thank you! I updated to this version yesterday specifically
for the "implicit localhost" behavior. It is very helpful since my
management machine has Python 3 installed at /usr/bin/python.

I did notice, though, that ansible-playbook seems to gather facts
between every play now, as opposed to once at the beginning like it did
in 1.4.3. Is this intended behavior? I don't see any mention of it in
CHANGELOG.md, but I could be overlooking something.

Regards,

--
♫Dustin
http://dustin.hatch.name/

[James Tanner]

Are you not setting gather_facts in each of your plays?

[Dag Wieers]

On Mon, 3 Mar 2014, James Tanner wrote:

> On 03/01/2014 09:16 PM, Dustin C. Hatch wrote:
>> On 02/28/2014 02:56 PM, Michael DeHaan wrote:
>> > Or should we say, it escapes!
>> >
>> > And it's on PyPi now.
>> >
>> > http://blog.ansibleworks.com/2014/02/28/ansible-1-5-released/
>> >
>> > Read the blog link above for details, and if you don't want to download
>> > via pip you can hit releases.ansible.com <http://releases.ansible.com>
>> > too.
>>

>> Great work, thank you! I updated to this version yesterday specifically
>> for the "implicit localhost" behavior. It is very helpful since my
>> management machine has Python 3 installed at /usr/bin/python.
>>
>> I did notice, though, that ansible-playbook seems to gather facts
>> between every play now, as opposed to once at the beginning like it did
>> in 1.4.3. Is this intended behavior? I don't see any mention of it in
>> CHANGELOG.md, but I could be overlooking something.
>

> Are you not setting gather_facts in each of your plays?

We noticed the same thing. Facts are gathered for every play, rather than
once per playbook.

The playbook does not mention gather_facts, nor did we specify anything in
/etc/ansible/ansible.cfg related to facts gathering.

--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, con...@dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]

[Dag Wieers]

On Mon, 3 Mar 2014, Dag Wieers wrote:

> On Mon, 3 Mar 2014, James Tanner wrote:
>
>> On 03/01/2014 09:16 PM, Dustin C. Hatch wrote:
>> > On 02/28/2014 02:56 PM, Michael DeHaan wrote:
>> > > Or should we say, it escapes!
>> > >
>> > > And it's on PyPi now.
>> > >
>> > > http://blog.ansibleworks.com/2014/02/28/ansible-1-5-released/
>> > >
>> > > Read the blog link above for details, and if you don't want to
>> > > download
>> > > via pip you can hit releases.ansible.com
>> > > <http://releases.ansible.com> too.
>> >
>> > Great work, thank you! I updated to this version yesterday specifically
>> > for the "implicit localhost" behavior. It is very helpful since my
>> > management machine has Python 3 installed at /usr/bin/python.
>> >
>> > I did notice, though, that ansible-playbook seems to gather facts
>> > between every play now, as opposed to once at the beginning like it did
>> > in 1.4.3. Is this intended behavior? I don't see any mention of it in
>> > CHANGELOG.md, but I could be overlooking something.
>>
>> Are you not setting gather_facts in each of your plays?
>
> We noticed the same thing. Facts are gathered for every play, rather than
> once per playbook.
>
> The playbook does not mention gather_facts, nor did we specify anything in
> /etc/ansible/ansible.cfg related to facts gathering.

Ok, I can see the change that was made. In the past gather_facts had 3
states: None, True and False. Now there's only True and False, and the
default is True, in which case "setup" is being run at the start of every
play as if "gather_facts: yes" was added.

[Brian Coca]

I went through this logic when attempting to add caching, the None state gathered facts only if the cache was empty (implied true), if the play had an explicit gather_facts: True it would 'refresh the cache'.

This was a nice optimization, removing it is making all my 'concatenated plays' a lot slower.

[James Tanner]

> --
> You received this message because you are subscribed to the Google
> Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ansible-project/CADn%2BHsww-EgUYfQjkMFQCkXMCw0%3DM7mWsBTZTPoyVT%3D4dzwCSg%40mail.gmail.com.

> For more options, visit https://groups.google.com/groups/opt_out.

https://github.com/ansible/ansible/commit/c039e276a2089cac804db4b0ea9072d8a27fee86

[Adam Morris]

We're pretty far off topic here, but I'm curious what you mean by plays versus playbooks here...

The documentation states that I can turn off fact gathering by explicitly setting gather_facts: no in my playbook...  I have not upgraded to 1.5 yet, so I haven't tested this, but I would want to do a gather_facts once at the beginning, but not for every task, or even role...

Adam

[Dag Wieers]

Apparently the "run setup for every play" is works-as-designed in 1.5.0.

https://github.com/ansible/ansible/pull/6251

But remember, this is not a democracy, and if it was, the silent majority
has already spoken ! ;-)

[Michael DeHaan]

It just happens once per play.

A playbook is a group of plays, in fact a list of them

A play takes a group of hosts and applies a list of roles and tasks to them

-- Michael

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/668e98c0-9503-4751-b612-51018f4b75e5%40googlegroups.com.

[Michael DeHaan]

This is not the question that the poster asked Dag (plays vs tasks) -
and I'm disappointed in the way you are approaching this discussion
from a position that assumes a shadow conspiracy.

Discuss a technical solution if you like but there is no need for the hyperbole.

-- Michael

> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/alpine.LRH.2.02.1403040157250.6649%40pikachu.3ti.be.

[Dag Wieers]

On Tue, 4 Mar 2014, Michael DeHaan wrote:

> This is not the question that the poster asked Dag (plays vs tasks) -
> and I'm disappointed in the way you are approaching this discussion
> from a position that assumes a shadow conspiracy.
>
> Discuss a technical solution if you like but there is no need for the hyperbole.

I approached this discussion with a patch to revert to the previous
behaviour.

The discussion is NOT plays vs tasks, again you have not understood what I
said in the issue ticket (or what bcoca said). ANd there is no shadow
conspiracy, it is mostly history repeating. You have a conviction and it
does not matter what people say. Every other view is a rare use-case.

Michael, you own this project and you give the direction. It doesn't
matter what I say or what I think. And if you wonder why I am frustrated,
THAT is exactly it. You make changes without even discussing it on this
list, and this is not the first time it happened and it won't be the last.
My patch for this regression was accepted, than reverted because nobody
knows what is a bug and what is design.

Your reason for this change is that it was never documented, and therefore
there is no specification to what is the correct behaviour. My position is
that how it worked for everyone the past 2 years is how people expect it
to behave. So the easiest solution is to document the existing behaviour,
instead of changing the behaviour.

So I come back to my remark, if the majority doesn't care and this is not
a democracy, the new way is works-as-designed. Have it your way !

So yes, I don't understand why you would want to undo the speed
improvements of 1.5.0 with a (default) mandatory facts gathering at each
play. Presumably because a task can change the facts. Well, if that's your
concern, you should force a facts gathering per task (because every task
can change facts and that has also not been documented). Or instead,
leave it up to the user to decide when to gather facts, but do it at
least once. (The way it always worked since Daniel Hokka Zakrisson
(dhozac) implemented it)

With the current change there is no guarantee that all systems have facts
gathered, so if you start ansible 1.5 with a:

----
- hosts: webservers
tasks:
- action: ping

- hosts: all
gather_facts: no
tasks:
- action: ping
----

You won't have facts except for the webservers. Whereas in the past
(pre 1.5) you could simply do:

----
- hosts: webservers
tasks:
- action: ping

- hosts: all
tasks:
- action: ping
---

And all systems would eventually have facts gathered one time. So total
control of facts gathering in 1.5 makes the default case worse, either
because you have facts gathered at every play (overhead), or because you
have to be careful in order to have facts gathered for all your hosts if
you would like to avoid the overhead.

If you keep up with the existing implementation in v1.5, I am going to
recommend people to always start with:

----
- hosts: all
gather_facts: no
tasks:
- action: setup

- hosts: webservers
gather_facts: no
tasks:
- action: ping
----

Simply to ensure you have facts for each host, and then disable facts
gathering on every subsequent play to avoid the additional overhead of
running the setup module.

I know I am not going to change your mind, but I care too much to not even
try to make the case (that you seem to ignoring).

[Michael DeHaan]

On Tue, Mar 4, 2014 at 8:03 AM, Dag Wieers <d...@wieers.com> wrote:

On Tue, 4 Mar 2014, Michael DeHaan wrote:

This is not the question that the poster asked Dag (plays vs tasks) -
and I'm disappointed in the way you are approaching this discussion
from a position that assumes a shadow conspiracy.

Discuss a technical solution if you like but there is no need for the hyperbole.

I approached this discussion with a patch to revert to the previous behaviour.

The discussion is NOT plays vs tasks, again you have not understood what I said in the issue ticket (or what bcoca said). ANd there is no shadow conspiracy, it is mostly history repeating. You have a conviction and it does not matter what people say. Every other view is a rare use-case.

Michael, you own this project and you give the direction. It doesn't matter what I say or what I think.

Actually it is does matter what everybody thinks to me.  Quite a lot!

I'll recap for the purposes of the list:

The question is removal of an undocumented and unclear flag, that modelled what was a "Tri-state" boolean.

In this case, if gather_facts was not explicitly set True, it was assumed "None", which meant "gather facts for hosts you have not talked to yet".   This means that any host that has registered something will be skipped in later plays, and can't have facts that are dynamic. 

This was an untuitive behavior, was never documented, and was removed.

 

And if you wonder why I am frustrated, THAT is exactly it. You make changes without even discussing it on this list, and this is not the first time it happened and it won't be the last.

Yes, this is true.

Ansible is our project, and we're not going to spin a wheel every time we want to change something and say "please may I".   However, we are going to use our combined experience of talking to thousands upon thousands of users, and do that to best help everyone here.

When that is done, if we occasionally screw up -- and we will -- it's best to ask how we can fix it, and that will result in a technical discussion.

 

My patch for this regression was accepted, than reverted because nobody knows what is a bug and what is design.

Something that has never been a documented feature, that actually breaks parts of Ansible, is not a regression.   But it's a fair concern, so read below...

 

Your reason for this change is that it was never documented, and therefore there is no specification to what is the correct behaviour. My position is that how it worked for everyone the past 2 years is how people expect it to behave. So the easiest solution is to document the existing behaviour, instead of changing the behaviour.

Perhaps.   if it's what we wanted, but I still don't feel it's the right behavior -- and it's always been the aim of the project to believe that if something documented is confusing, the feature is confusing, and it shouldn't be done that way -- fix the system until the docs are *EASY*.

I'll be straight though.  Ansible was a different project two years ago.   I was running things as a project in my spare time, and I was *MUCH* more inclined to auto-merge things that people wanted.   

In the past (and they are great people), Daniel and Stephen also had direct commit access.   That isn't true now and we review things closer, we care about consistency closer, and we care about code a bit more -- that's not a knock at them, that just means we are taking things a lot more seriously than when this was a "nights and weekends" project.

In this case the code in question was causing problems -- and it was undocumented behavior that would be confusing to most -- and also it was overloading a boolean in a way that was VERY confusing and wouldn't allow anyone using the boolean to rely on the truthiness of the boolean.

 

So I come back to my remark, if the majority doesn't care and this is not a democracy, the new way is works-as-designed. Have it your way !

So yes, I don't understand why you would want to undo the speed improvements of 1.5.0 with a (default) mandatory facts gathering at each play. Presumably because a task can change the facts. Well, if that's your concern, you should force a facts gathering per task (because every task can change facts and that has also not been documented). Or instead, leave it up to the user to decide when to gather facts, but do it at least once. (The way it always worked since Daniel Hokka Zakrisson (dhozac) implemented it)

The gather_facts on each play is still not mandatory, as you can have a explicit

gather_facts: "no" on each play.

What I didn't like was how gather_facts <previous implementation> broke the ability to gather new facts, which is a serious concern, and how it relied on SETUP_CACHE to make that decision, which is also sketchy.

 

With the current change there is no guarantee that all systems have facts gathered, so if you start ansible 1.5 with a:

----
- hosts: webservers
  tasks:
  - action: ping

- hosts: all
  gather_facts: no
  tasks:
  - action: ping
----

You won't have facts except for the webservers. Whereas in the past (pre 1.5) you could simply do:

----
- hosts: webservers
  tasks:
  - action: ping

- hosts: all
  tasks:
  - action: ping
---

I understand the above and appreciate the technical discussion.   To be clear, it could be done like it was intended with:

- hosts: all  # or whatever group of systems where this is needed

  gather_facts: True  # default

  tasks: []

- hosts: webservers

  gather_facts: False

  tasks:

      - some_meaningful_action

And this also has the benefit of removing the ping step.   It will also perform the exact same number of tasks, and I think it's also more explicit, which is also Ansible's nature.

 

And all systems would eventually have facts gathered one time. So total control of facts gathering in 1.5 makes the default case worse, either because you have facts gathered at every play (overhead), or because you have to be careful in order to have facts gathered for all your hosts if you would like to avoid the overhead.

 

If you keep up with the existing implementation in v1.5, I am going to recommend people to always start with:

----
- hosts: all
  gather_facts: no
  tasks:
  - action: setup

This can be written a lot more clearly with gather_facts: yes and a null tasks block.

 

- hosts: webservers
  gather_facts: no
  tasks:
  - action: ping
----

Simply to ensure you have facts for each host, and then disable facts gathering on every subsequent play to avoid the additional overhead of running the setup module.

I know I am not going to change your mind, but I care too much to not even try to make the case (that you seem to ignoring).

See, this is what I hate here -- you're jumping to assumptions about me already.

I'm not against providing a *better* way to solve this problem, that's more clear and explicit, but the existing hack (being default behavior) was breaking things and was not a good way to go.

What about simply looking at this as "gather_facts: auto" being a new mode, and also being able to set that default in ansible.cfg?

That's the kind of technical discussion I like to have.

We're going to disagree from time, but I think many people know I can be persuaded with logic.  In this case the existing behavior is a 'secret' of a patch and not one I agree with, but it's not saying we can't have similar functionality exposed in clearer ways.

 

--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, con...@dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/alpine.LRH.2.02.1403041339050.6649%40pikachu.3ti.be.

[Brian Coca]

the gather_facts: false in subsequent plays works if they are all in the same playbook.  But it breaks when doing (and all plays using facts):

- include: playA.yml

- include: playB.yml

- include: playC.yml

Which is a pretty common setup, specially for site.yml and still allows you to run the plays individually. From irc and tickets this is a pretty common use case, I understand that it is undocumented/unclear behavior but I think a better change is through documentation vs removing the way this has worked in the experience of the majority of the users.

Also I have not heard about issues with facts not being refreshed, aside from add_host, group_by which now specifically force a gather/cache invalidation. 

I think you are 'fixing' it for the rare case, which could also be solved easier by just documenting the feature or adding a default 'cached' option that would be clear, explicit and still work as it has (which is pretty optimal for 99% of the usage).

[Serge van Ginderachter]

On 4 March 2014 14:17, Michael DeHaan <mic...@ansible.com> wrote:

This means that any host that has registered something will be skipped in later plays, and can't have facts that are dynamic. 

​I wasn't aware of this, whilst I now understand​ why it mishappened that way.

I guess this is the key part in this change.

It would have solved a lot of questions though if this would have been made more clear. Like e.g. when Dag's merged patch was reverted.

Given the big change in behavior, at least a notice on the mailing list and a mention in the CHANGELOG would have been a good practice.

Now all back to work :)

     Serge

"All problems are caused, at some level, by poor communication."

[Michael DeHaan]

"the gather_facts: false in subsequent plays works if they are all in the same playbook.  But it breaks when doing (and all plays using facts):"

Are you sure?

Facts within the same Ansible run (different plays) are not cleared.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsy17_huMRciD5hOfzPUqn%2BMQFNFaSfGCLcEzDRDq6tJ%2Bg%40mail.gmail.com.

[Brian Coca]

not what I meant, I meant that the gather_facts:false doesn't work for this case as when those plays get executed individually, they won't have facts. If left to true, then when executing in same run they will all trigger a fact gathering even if not needed.

[Michael DeHaan]

They would if the first play was an empty fact gathering play.

(I understand what you are getting at with not having one)

On Tue, Mar 4, 2014 at 8:46 AM, Brian Coca <bria...@gmail.com> wrote:

not what I meant, I meant that the gather_facts:false doesn't work for this case as when those plays get executed individually, they won't have facts. If left to true, then when executing in same run they will all trigger a fact gathering even if not needed.

> Facts within the same Ansible run (different plays) are not cleared.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHswLPYjsG-HCtcRdChPwZMyu-fXY_Lv9RKfgnDt3BgvnHg%40mail.gmail.com.

[Brian Coca]

er ... no ... you lost me here.

[Michael DeHaan]

I'm saying this works for top level playbooks, and that facts are not erased between includes:

# site.yml

- hosts: all

  gather_facts: yes

  tasks: []

- include: webserver_plays.yml

- include: dbserver_plays.yml

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsxa9%3Dq1_dPZN_mr39T0VagY8QGt-oFJQuZDqMsXOwO9vA%40mail.gmail.com.

[Dag Wieers]

On Tue, 4 Mar 2014, Michael DeHaan wrote:

> On Tue, Mar 4, 2014 at 8:03 AM, Dag Wieers <d...@wieers.com> wrote:
>> On Tue, 4 Mar 2014, Michael DeHaan wrote:

> I'll recap for the purposes of the list:
>
> The question is removal of an undocumented and unclear flag, that modelled
> what was a "Tri-state" boolean.
>
> In this case, if gather_facts was not explicitly set True, it was assumed
> "None", which meant "gather facts for hosts you have not talked to yet".
> This means that any host that has registered something will be skipped in
> later plays, and can't have facts that are dynamic.

What you call "dynamic facts" is simply rerunning setup at the start of
every play. But they are not dynamic because the next task won't have them
either.

In most if not all cases facts are static, and in case you want to
re-enforce facts gathering, you could simply enable that for a specific
play. Or even better, run the setup module within a play. There is no need
for re-running the setup module at the start of every play. The overhead
is big and the opportunities are scarce, especially if you properly
structure your playbooks with self-contained plays.

> This was an untuitive behavior, was never documented, and was removed.

It was not unintuitive, everyone understood this well, nobody complained
about it and could have been documented (like many other things that are
not documented yet).

>> And if you wonder why I am frustrated, THAT is exactly it. You make
>> changes without even discussing it on this list, and this is not the first
>> time it happened and it won't be the last.
>
> Yes, this is true.
>
> Ansible is our project, and we're not going to spin a wheel every time we
> want to change something and say "please may I". However, we are going to
> use our combined experience of talking to thousands upon thousands of
> users, and do that to best help everyone here.
>
> When that is done, if we occasionally screw up -- and we will -- it's best
> to ask how we can fix it, and that will result in a technical discussion.

I asked it on IRC, send in a fix and it was accepted and reverted.

BTW That technical discussion was not held, the decision was made when you
committed it for v1.5 and released it as part of v1.5. Reverting now is
harder because suddenly this is the default in v1.5.

What's even more, it was not even announced as a change in the ChangeLog.

>> So I come back to my remark, if the majority doesn't care and this is not
>> a democracy, the new way is works-as-designed. Have it your way !
>>
>> So yes, I don't understand why you would want to undo the speed
>> improvements of 1.5.0 with a (default) mandatory facts gathering at each
>> play. Presumably because a task can change the facts. Well, if that's your
>> concern, you should force a facts gathering per task (because every task
>> can change facts and that has also not been documented). Or instead, leave
>> it up to the user to decide when to gather facts, but do it at least once.
>> (The way it always worked since Daniel Hokka Zakrisson (dhozac) implemented
>> it)
>>
>
> The gather_facts on each play is still not mandatory, as you can have a
> explicit
>
> gather_facts: "no" on each play.
>
> What I didn't like was how gather_facts <previous implementation> broke the
> ability to gather new facts, which is a serious concern, and how it relied
> on SETUP_CACHE to make that decision, which is also sketchy.

Well, my solution would have looked very differently. To me it is a design
fault if variables you register end up in the SETUP_CACHE (!). That's
very alarming to me (not knowing that part of the code).

But a simple boolean to keep track whether setup was run could have been
sufficient. In the ansible-provisioning project we actually have different
namespaces for different facts modules, and there is a special fact
module_<name> which is set if that specific module has run once. Such a
mechanism could have been sufficient to fix this corner case.

BTW In the discussion of pull request you never mentioned what exact
problem was fixed by removing the old behaviour. It could help if commits
would give a proper description to why it was needed, and what it changes.
SOmething you expect from external pull requests anyway.

>> Simply to ensure you have facts for each host, and then disable facts
>> gathering on every subsequent play to avoid the additional overhead of
>> running the setup module.
>>
>> I know I am not going to change your mind, but I care too much to not even
>> try to make the case (that you seem to ignoring).
>
> See, this is what I hate here -- you're jumping to assumptions about me
> already.

Like you didn't do about me, right ? :)

> I'm not against providing a *better* way to solve this problem, that's more
> clear and explicit, but the existing hack (being default behavior) was
> breaking things and was not a good way to go.
>
> What about simply looking at this as "gather_facts: auto" being a new mode,
> and also being able to set that default in ansible.cfg?

How is that any different as it was. You complained about gather_facts
having three states before and it not being a boolean. And now you propose
to make it a three-state entity again ?

> That's the kind of technical discussion I like to have.

Sure, BEFORE you release something as ansible v1.5.

You are ignoring the fact that you committed it and released it in a
new release, only then when people complain to give the notion you prefer
to have a proper technical discussion about it.


Sure gather_facts: auto is fine by me. But it would have been a lot better
if that was what shipped with Ansible v1.5 in the first place. Now it
simply looks as nobody has real control over what's going on unless you
give your final say. And then we have the technical discussion...

[Brian Coca]

I understand that, it is not the problem.

Consider my site.yml which has 23 different plays. To run with the new behavior I gather facts 23 times.

If I change my included plays to gather_facts: false to avoid this, I cannot run them individually anymore if they need facts (most of them do).

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEVJ8QOuTOoM8jzRXq_gLL7TS%2BJ%2BFiWbhnaOO-vhVbQKwcmkbg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Brian Coca
Stultorum infinitus est numerus
0110000101110010011001010110111000100111011101000010000001111001011011110111010100100000011100110110110101100001011100100111010000100001

Pedo mellon a minno

[Serge van Ginderachter]

On 4 March 2014 14:54, Michael DeHaan <mic...@ansible.com> wrote:

I'm saying this works for top level playbooks, and that facts are not erased between includes:

# site.yml

- hosts: all

  gather_facts: yes

  tasks: []

- include: webserver_plays.yml

- include: dbserver_plays.yml

What I have been doing until now as a personal best practice, is always explicitly set gather_facts: true or false on every play, setting it on true in plays where facts are used.

This way facts gathering would trigger at the very last when I need them. Now this new behavior will trigger extra runs, burning unneeded cycles.

I should also agree with Brian, I never encountered the key problem that is solved here though.

On a dangerous side node, I actually never understood why registered variables and set_fact vars live in the same namespace as gathered facts.

  Serge

[Brian Coca]

Is it not what  "module_setup": "true", was for?

[Dag Wieers]

On Tue, 4 Mar 2014, Brian Coca wrote:

> I understand that, it is not the problem.
>
> Consider my site.yml which has 23 different plays. To run with the new
> behavior I gather facts 23 times.
>
> If I change my included plays to gather_facts: false to avoid this, I
> cannot run them individually anymore if they need facts (most of them do).

Exactly, this is what we do often as well. I didn't this implication.

But now that Michael proposes to have gather_facts: auto configurable from
ansible.cfg, it would suffice for me. What he thinks is best by default is
not my concern, but his customer's and support's concern.

That said, I would definitely fix the logic that validates from
SETUP_CACHE whether setup was already run, and make a special boolean or
fact to keep track of whether it has run. Which IMO is the real bug that
needs fixing.

[Michael DeHaan]

On Tue, Mar 4, 2014 at 8:58 AM, Dag Wieers <d...@wieers.com> wrote:

On Tue, 4 Mar 2014, Michael DeHaan wrote:

On Tue, Mar 4, 2014 at 8:03 AM, Dag Wieers <d...@wieers.com> wrote:

On Tue, 4 Mar 2014, Michael DeHaan wrote:

I'll recap for the purposes of the list:

The question is removal of an undocumented and unclear flag, that modelled
what was a "Tri-state" boolean.

In this case, if gather_facts was not explicitly set True, it was assumed
"None", which meant "gather facts for hosts you have not talked to yet".
This means that any host that has registered something will be skipped in
later plays, and can't have facts that are dynamic.

What you call "dynamic facts" is simply rerunning setup at the start of every play. But they are not dynamic because the next task won't have them either.

I was referring to things like facts.d

And that they aren't gathered between tasks is not so important, but the fact that they would be intuitively re-gathered if you wrote another play.

Most people do not set "gather_facts" behavior on a play at all, because gathering facts is not expensive to them.

Brian in particular has some massively overloaded machines -- which, should there be an ASPCA for computer machines, would be recieving some industry protection.

Most folks also run with sufficiently high forks values :)

 

In most if not all cases facts are static, and in case you want to re-enforce facts gathering, you could simply enable that for a specific play. Or even better, run the setup module within a play. There is no need for re-running the setup module at the start of every play. The overhead is big and the opportunities are scarce, especially if you properly structure your playbooks with self-contained plays.

This was an untuitive behavior, was never documented, and was removed.

It was not unintuitive, everyone understood this well, nobody complained about it and could have been documented (like many other things that are not documented yet).

I'd definitely disagree that everyone knew about it -- it wasn't clear even to *ME*.

 

And if you wonder why I am frustrated, THAT is exactly it. You make
changes without even discussing it on this list, and this is not the first
time it happened and it won't be the last.

Yes, this is true.

Ansible is our project, and we're not going to spin a wheel every time we
want to change something and say "please may I".   However, we are going to
use our combined experience of talking to thousands upon thousands of
users, and do that to best help everyone here.

When that is done, if we occasionally screw up -- and we will -- it's best
to ask how we can fix it, and that will result in a technical discussion.

I asked it on IRC, send in a fix and it was accepted and reverted.

To be clear, that was merged by James Tanner, and we wasn't aware of the previous change.

This was a communication error between us and not his fault :)

 

BTW That technical discussion was not held, the decision was made when you committed it for v1.5 and released it as part of v1.5. Reverting now is harder because suddenly this is the default in v1.5.

What's even more, it was not even announced as a change in the ChangeLog.

That was my error.

The changelog is frequently updated retroactively and will continue to be on many occasions.   Also due to the extreme pace at which this project moves there are times where we're going to miss something in the changelog.

 

So I come back to my remark, if the majority doesn't care and this is not
a democracy, the new way is works-as-designed. Have it your way !

So yes, I don't understand why you would want to undo the speed
improvements of 1.5.0 with a (default) mandatory facts gathering at each
play. Presumably because a task can change the facts. Well, if that's your
concern, you should force a facts gathering per task (because every task
can change facts and that has also not been documented). Or instead, leave
it up to the user to decide when to gather facts, but do it at least once.
(The way it always worked since Daniel Hokka Zakrisson (dhozac) implemented
it)

The gather_facts on each play is still not mandatory, as you can have a
explicit

gather_facts: "no" on each play.

What I didn't like was how gather_facts <previous implementation> broke the
ability to gather new facts, which is a serious concern, and how it relied
on SETUP_CACHE to make that decision, which is also sketchy.

Well, my solution would have looked very differently. To me it is a design fault if variables you register end up in the SETUP_CACHE (!). That's
very alarming to me (not knowing that part of the code).

I'm not sure why this should be alarming to you.

 

But a simple boolean to keep track whether setup was run could have been sufficient.

Perhaps, though the default state should be to gather facts.   Like I've mentioned below I'd accept a "gather_facts: auto" and a configurable default in ansible.cfg

 

BTW In the discussion of pull request you never mentioned what exact problem was fixed by removing the old behaviour. It could help if commits would give a proper description to why it was needed, and what it changes. SOmething you expect from external pull requests anyway.

Now this is just getting nitpicky about commit messages.

 

Simply to ensure you have facts for each host, and then disable facts
gathering on every subsequent play to avoid the additional overhead of
running the setup module.

I know I am not going to change your mind, but I care too much to not even
try to make the case (that you seem to ignoring).

See, this is what I hate here -- you're jumping to assumptions about me
already.

Like you didn't do about me, right ? :)

I didn't.

I'm not against providing a *better* way to solve this problem, that's more
clear and explicit, but the existing hack (being default behavior) was
breaking things and was not a good way to go.

What about simply looking at this as "gather_facts: auto" being a new mode,
and also being able to set that default in ansible.cfg?

How is that any different as it was. You complained about gather_facts having three states before and it not being a boolean. And now you propose to make it a three-state entity again ?

That's a valid point.    Perhaps it should be a new setting altogether.    

gather_facts:  True/False

It may also be better addressed by fact caching.

 

That's the kind of technical discussion I like to have.

Sure, BEFORE you release something as ansible v1.5.

You are ignoring the fact that you committed it and released it in a new release, only then when people complain to give the notion you prefer to have a proper technical discussion about it.

I'm always open to a discussion.   And I don't really view it as complaint -- I see it as a need to have a discussion about what the project/code needs to be doing.

This is what this list is for.

 

Sure gather_facts: auto is fine by me. But it would have been a lot better if that was what shipped with Ansible v1.5 in the first place. Now it simply looks as nobody has real control over what's going on unless you give your final say. And then we have the technical discussion...

I'll say this, every time this kind of attitude comes up, it makes me want to help you less and less.

And it takes time out of when we could be helping others.

 

--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, con...@dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/alpine.LRH.2.02.1403041422360.6649%40pikachu.3ti.be.

[Michael DeHaan]

On Tue, Mar 4, 2014 at 9:09 AM, Dag Wieers <d...@wieers.com> wrote:

On Tue, 4 Mar 2014, Brian Coca wrote:

I understand that, it is not the problem.

Consider my site.yml which has 23 different plays. To run with the new
behavior I gather facts 23 times.

If I change my included plays to gather_facts: false to avoid this, I
cannot run them individually anymore if they need facts (most of them do).

Exactly, this is what we do often as well. I didn't this implication.

But now that Michael proposes to have gather_facts: auto configurable from ansible.cfg, it would suffice for me. What he thinks is best by default is not my concern, but his customer's and support's concern.

I'd like to let this thread drop, but I don't want to leave this assumption.   Commerce has not even been a part of this discussion.   What we do on this list is try to maintain Ansible for thousands upon thousands of users, and try to make sure that we design and test things to ensure we create a good product for everyone.

We are not, by any means, responding to a customer ticket to fix behavior.

However, I *AM* going to use my expertise about how to make Ansible easier to everyone from time to time, to try and target Ansible so that it helps the most people.

Which if people like Ansible to date, I think they have to trust that is what we have been doing, and are continuing to do.   

It's not going to be the case that the most vocal always get to win their arguments, as that is a path to disaster for everyone involved.  

When people have concerns -- PLEASE -- express them.  But also spare the allegations of evil-doing.

The transmuto-ray in the basement isn't even ready yet.

 

[Brian Coca]

@micheal, I'm not even making the case for my machines, my fact cache plugin has solved that issue temporarily until I get them off life support (1/2 way there).

About the boolean that shows setup has run, IT ALREADY EXISTS and we ALREADY CHECK IT:   "module_setup": "true"

which is set ONLY by the setup plugin to indicate that it ran, this distinguishes it from registered vars so I'm not sure how the problem you are trying to fix occurred.

[Michael DeHaan]

Here was the code in question, which didn't check it.   It probably should have.   In either case, I'm open to having a different flag to express this though we should be avoiding tri-state booleans in the code, which is a question of how it should be expressed interface wise.

Probably a policy variable in ansible.cfg could just control what "gather_facts: True" does, and there's always the explicit call to setup if you want it.   So the "None" thing can still go.

This is the heinous line in question:

- host_list = [h for h in host_list if h not in self.SETUP_CACHE or 'module_setup' not in self.SETUP_CACHE[h]]

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHsz8mpbE5G3KUpt-vYsYHbeKBewGznP0hkyr%3Dk68oRKZ7Q%40mail.gmail.com.

[Dag Wieers]

On Tue, 4 Mar 2014, Michael DeHaan wrote:

> When people have concerns -- PLEASE -- express them. But also spare the
> allegations of evil-doing.

You read too much into what I say. I am simply saying that a change like
this affects you much more than it does me. I can patch it myself, or
customize the default behaviour, whereas you have to live with the support
of it at a much larger scale.

Maybe I am also saying that I don't want to support the possible fallout
of such changes in the community, but that's self-evident. Do leave any
paranoid thoughts out of it please, not good for the heart :)

[Michael DeHaan]

I suggest we now Fedex beverages to each other.

note:  may have been spending last several years building up immunity to iocaine powder :)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/alpine.LRH.2.02.1403041551100.6649%40pikachu.3ti.be.

[Dag Wieers]

On Tue, 4 Mar 2014, Brian Coca wrote:

> Is it not what "module_setup": "true", was for?

Now I feel stupid :)

https://github.com/ansible/ansible/commit/fedfd187749654105a22be20c27e0050bc722d0a

But yes, that's the logic we (I guess) copied however I didn't see the
setup module exposing this. So what confused me here is that this is
piggy-backed by the facts-gathering logic, rather than part of the setup
module as I was used to.

So in my opinion, a call to setup (through an action) should expose this
as well. My bad :)

[Dag Wieers]

On Tue, 4 Mar 2014, Dag Wieers wrote:

> On Tue, 4 Mar 2014, Brian Coca wrote:
>
>> Is it not what "module_setup": "true", was for?
>
> Now I feel stupid :)
>
> https://github.com/ansible/ansible/commit/fedfd187749654105a22be20c27e0050bc722d0a
>
> But yes, that's the logic we (I guess) copied however I didn't see the setup
> module exposing this. So what confused me here is that this is piggy-backed
> by the facts-gathering logic, rather than part of the setup module as I was
> used to.
>
> So in my opinion, a call to setup (through an action) should expose this as
> well. My bad :)

Ok, so this pull-request implements the setup module how it should have
been from the start:

https://github.com/ansible/ansible/pull/6278

No special magic from the playbook/__init__.py, and it behaves like other
facts modules, especially when run as a task.

[Brian Coca]

I hope this satisfies all parties

https://github.com/ansible/ansible/pull/6508

[Michael DeHaan]

Perhaps some assignment of play.gather_facts to the effective boolean, if None, might simplify things?

That way any code in question that might refer to it can just check play.gather_facts and get the mode.

But yeah, this config setting is, I think, exactly the way to do it to make the system self documenting.

Perhaps some renames though?

always => implicit

never => explicit

smart => smart

never might tend to imply gather_facts might be ignored.  Some of this could also be dealt with in comments in the ansible.cfg though

(Also, green bikesheds!)

On Sat, Mar 15, 2014 at 4:25 PM, Brian Coca <bria...@gmail.com> wrote:

I hope this satisfies all parties

https://github.com/ansible/ansible/pull/6508

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADn%2BHswJiSC00qi6oNsoNEF4tcfAKz8YjRfvAqnQdxJKDCG6%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Coca]

I'll make a resubmit with the changes ... but the font is going to be yellow!

[Brian Coca]

I changed the settings, but kept the trinary boolean (I know it bugs you) as I think it is the simplest implementation.

[Michael DeHaan]

We'll take a look.

It's probably better if we can get this to have predicate methods like "play.should_gather_facts()"  or something if we want to escape the need for having to repeat test-logic for it in the code every time this field is accessed.  But that can wait.

On Sat, Mar 15, 2014 at 6:15 PM, Brian Coca <bria...@gmail.com> wrote:

I changed the settings, but kept the trinary boolean (I know it bugs you) as I think it is the simplest implementation.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/e6739ee6-d363-4a3c-927d-9ea0ce2a9572%40googlegroups.com.