Python API Documentation

[Michael Mattioli]

Documentation for the Python API is very sparse; aside from this page, there's really nothing else to offer aside from "read the source code and figure it out". While the source code is generously commented, it would be nice to have some official documentation and detailed examples of API usage as was done for every other piece of Ansible. Sure, it's feasible to get something working by just poking around and seeing what works and what doesn't but it's much more beneficial to have official documentation so that developers are aware of best practices (i.e. use this class "X" instead of class "Y" if trying to accomplish "Z" because "A, B, and C") and optimal use of the API. Given that 2.0 was released fairly recently and it's mentioned that API usage is different in 2.0, I can understand if API documentation was put on hold as to avoid re-writing it a second time but it'd be nice to know if/when something like this will be released/written.

[Brian Coca]

IF you check ,there wasn't much more for v1 API. The API is secondary
to the usage of Ansible command line tools, we are lagging behind in a
lot of other documentation that will be a priority over this.

That said, we will consider pull requests that add this documentation.

--
Brian Coca

[Alexander Sauta]

Hi.
The v2 API is much more complex and it's really unclear how to customize it. Maybe you guys could provide some quick tips here?
E.g., I don't get how to use DataLoader and VariableManager objects. In example on the site there is nothing that explains how it all works together with TaskQueueManager and Play.

My particular task is to apply ansible roles to hosts from inventory stored in database. Or more specifically: how to feed inventory and variables from python dict into to v2 API objects. If you could give me the tips I could probably write a short documentation article about it.

--
Alex

понедельник, 1 февраля 2016 г., 18:05:57 UTC+3 пользователь Brian Coca написал:

[Timothy Appnel]

You should be aware that only the plugin APIs are guaranteed to work from version to version. All the things you listed are not part of the plugin API.

Timothy Appnel Senior Architect Ansible, Inc. • www.ansible.com

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexander Sauta]

I'm confused, what is the recommended method of calling ansible programmatically from python scripts then? From the API documentation on ansible.com I got the impression that one is supposed to use 2.0 API.

пятница, 5 февраля 2016 г., 2:15:59 UTC+3 пользователь Timothy Appnel написал:

[Brian Coca]

The API is there for use by the Ansible command line, it is not the recommended way to use Ansible. It is shipped as part of the Ansible toolset and was not intended for direct use,  but since some users insisted on it we added that single documentation page.  

--

Brian Coca

[Serge van Ginderachter]

On 5 February 2016 at 15:33, Brian Coca <bc...@ansible.com> wrote:

It is shipped as part of the Ansible toolset and was not intended for direct use,

​Why would that be?​

[Matt Martz]

Although it is primary to the operation of Ansible, it is there because the Ansible CLI tools need it to be.  While backwards compat will exist for the CLI tools, underlying APIs may need to change, in backwards incompatible ways.  At this time, my feeling is there is already enough to have to worry with, without needing to worry that every entry point into the code base maintains backwards compat.

Not to mention the licensing side of things.  Any software you write that imports Ansible must also use the GPL license.

If we go by Tower as the recommended way to integrate with Ansible, then shelling out to Ansible is the appropriate way.

Some of this may change in the future, but I don't think we are at a place where we can or should recommend the use of the API as a first class interface to Ansible.

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Serge van Ginderachter]

​​Matt, are you speaking on behalf of the core team? I'm not sure how to understand the context of this​.

On 5 February 2016 at 15:55, Matt Martz <ma...@sivel.net> wrote:

Although it is primary to the operation of Ansible, it is there because the Ansible CLI tools need it to be.  While backwards compat will exist for the CLI tools, underlying APIs may need to change, in backwards incompatible ways. 

​To be honest, this is a bit triggering a 'WTF' to me. Perhaps I misunderstand your words.​ Why do you think that is something we need to accept?

 

At this time, my feeling is there is already enough to have to worry with, without needing to worry that every entry point into the code base maintains backwards compat.

​There sure are things to worry about, but the API​

 

​is IMHO part of it. I'm not sure why you think this way.​

Not to mention the licensing side of things.  Any software you write that imports Ansible must also use the GPL license.

​And that is a problem because...? That's what the GPL is all about. You make that sound like a bad thing?

​

 

If we go by Tower as the recommended way to integrate with Ansible, then shelling out to Ansible is the appropriate way.

​Let's face it, Tower "integrates" by forking a process, because of the GPL, to be able to keep it (for now) proprietary. There's nothing wrong with that, but using that reasoning to say "don't use Ansible as an API" is a bit off to me.

Some of this may change in the future, but I don't think we are at a place where we can or should recommend the use of the API as a first class interface to Ansible.

​As things are for now, using the API is a problem, for sure. 

That is however an effect of a deeper cause

​, not just some problem by itself.​

  Serge​

 

[Matt Martz]

These opinions expressed are solely my own and do not express the views or opinions of etc, etc, etc...

[Timothy Appnel]

Working for Ansible, but not being on the core team, I have the same understanding as sivel here. (Including the GPL licensing part hence Tower's architecture of utilizing as a subprocess instead of an import.) The core team puts effort in to not breaking compatibility, but they reserve the right to break something if it serves the great good of the software's intended uses. So have at it, but be aware of what you are getting in to.

I believe something to this effect is going to be added to that doc page to clarify.

<tim/>

Timothy Appnel Senior Architect Ansible, Inc. • www.ansible.com

--

[Serge van Ginderachter]

On 6 February 2016 at 03:28, Abhijit Menon-Sen <a...@2ndquadrant.com> wrote:

> I think we need to accept it because it's reality.
>
> The focus of the project is to produce useful command-line tools, and I
> think that's the correct focus in terms of maximising utility.
>
> I would like to see improvements in (a) being able to run ansible as a
> subprocess, and (b) internal interfaces to make it easier to, say, set
> up and invoke a PlaybookExecutor. If we do these things, we may end up
> with a useful API someday, but I certainly don't want to *start* from
> there, i.e., promise that the present interface will remain unchanged
> forever.
>
> Maintaining a stable API for a project like Ansible is a non-trivial
> effort. If someone is brave enough to tackle that task (of coming up
> with a high-level API that can be retrofitted without major disruption,
> making it maintainable, and then actually documenting and maintaining
> it in the long term)… but I don't think we're anywhere near that point.

I guess my understanding of the lower level code lacks in being up to
date, and will stand corrected for now.

To me that is however are game changer of how I have been looking at
Ansible, and what I expected from it.

> P.S. Serge, your mail is full of U+8203 (zero width spaces) at the start
> and end of every line.

No clue. I was using the GMail web composer...


Serge

[Abhijit Menon-Sen]

At 2016-02-06 13:20:38 +0100, se...@vanginderachter.be wrote:
>
> To me that is however are game changer of how I have been looking at
> Ansible, and what I expected from it.

Just to be entirely clear, if I needed to use the Ansible API (which I
currently do not, but could certainly imagine reasons to do), I would
use it and try to improve it (and be prepared to put in the effort to
adapt to any future interface changes).

But I think it would be a mistake to declare the *current* API to be
stable, when it is quite low-level, not designed explicitly as an API
for external users, and—most significantly—has no real external users.

-- Abhijit

[Abhijit Menon-Sen]

At 2016-02-05 18:31:38 +0100, se...@vanginderachter.be wrote:
>
> On 5 February 2016 at 15:55, Matt Martz <ma...@sivel.net> wrote:
> >
> > Although it is primary to the operation of Ansible, it is there
> > because the Ansible CLI tools need it to be. While backwards compat
> > will exist for the CLI tools, underlying APIs may need to change, in
> > backwards incompatible ways.
>
> To be honest, this is a bit triggering a 'WTF' to me. Perhaps I
> misunderstand your words. Why do you think that is something we
> need to accept?

I think we need to accept it because it's reality.

The focus of the project is to produce useful command-line tools, and I
think that's the correct focus in terms of maximising utility.

I would like to see improvements in (a) being able to run ansible as a
subprocess, and (b) internal interfaces to make it easier to, say, set
up and invoke a PlaybookExecutor. If we do these things, we may end up
with a useful API someday, but I certainly don't want to *start* from
there, i.e., promise that the present interface will remain unchanged
forever.

Maintaining a stable API for a project like Ansible is a non-trivial
effort. If someone is brave enough to tackle that task (of coming up
with a high-level API that can be retrofitted without major disruption,
making it maintainable, and then actually documenting and maintaining
it in the long term)… but I don't think we're anywhere near that point.

-- Abhijit

[Jason Edelman]

Brian,

Using the example below, how do you access the JSON return data from each task?  In the past, there was ansible.runner and ansible.playbook.  In the runner object, b/c it was task/module-based I was able to get the return data no problem and use the playbook object to run pre-built playbooks.  The new API in 2.0 seems to be more like ansible.playbook but is being compared to runner in the docs.

All that seems available in tqm._stats is summary information, if a device was contacted, if it was "ok", etc.  I'm looking for the data you see when you run playbook in verbose mode.

Any pointers?

Thanks,
Jason

tqm = None
try:
    tqm = TaskQueueManager(
              inventory=inventory,
              variable_manager=variable_manager,
              loader=loader,
              options=options,
              passwords=passwords,
              stdout_callback='default',
          )
    result = tqm.run(play)
finally:
    if tqm is not None:
        tqm.cleanup()

[Matt Martz]

When using tqm the only way to get access to that info is to write a callback plugin to capture it.  Otherwise you would need to use TaskExecutor directly, which means you have to be responsible for looping the hosts and tasks yourself.

--

You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Edelman]

Thanks..makes sense.  Can you share an example of it being used?  Not fun trying to figure out what data type each var in init needs to be.

Thanks,
Jason

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Toshio Kuratomi]

http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/scripts/builders/isbuilding.v2

I hacked together a callback strategy a while back. It's based one
level away from TQM but IIRC, the callback plugin can be passed to
TQM's __init__() in the same way as it can be passed to the higher
level.

-Toshio

>>> email to ansible-deve...@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Matt Martz
>> @sivel
>> sivel.net
>>

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to ansible-deve...@googlegroups.com.

[Alexander Sauta]

Is there a way to pass the callback object in similar way when executing playbook?
Class PlaybookCLI doesn't seem to honor `callback` parameter so I was able to make it use callbacks only using this code:

    pbex = PlaybookExecutor(playbooks=playbooks, inventory=inventory,
                            variable_manager=variable_manager,
                            loader=loader, options=options,
                            passwords=passwords)
    cb = ResultAccumulator()
    pbex._tqm._stdout_callback = cb
    results = pbex.run()

Obviously, this is not a great way to do it, so probably you or someone else might propose a better solution?

Alexander.

вторник, 9 февраля 2016 г., 1:59:33 UTC+3 пользователь tkuratomi написал:

[Matt Martz]

I am not aware of any better way really.  As you can see from various conversations here, a high level API for users to interact with doesn't really exist, and this wasn't the primary goal during the API design.

Your way, is the best you can probably achieve at the moment, and in fact was what I was going to recommend before seeing your code sample.

Just do be aware that there have been discussions on re-evaluating all places where "private" attributes are used and declared, to determine if they would be better as "public" attributes.  So things could change in the future.

[Alexander Sauta]

Ok, got it, thank you for your reply.
btw, why is it called stdout_callback? What's the difference/purpose between stdout_callback and _callback_plugins in TQM?

Alexander.

четверг, 11 февраля 2016 г., 19:57:35 UTC+3 пользователь Matt Martz написал:

[Matt Martz]

Per it's name, it was defined, to only load a single callback that defines itself as 'stdout'.  Generally speaking the name is designed around the idea that the bundled command line tools are the users of the API.

At minimum a "stdout" callback is required.

You can kind of think of it as the "default" callback.  All other callbacks are in addition to this default callback.

[Alexander Sauta]

Ok, thank you. So I'm gonna define all callback methods in _stdout_callback and this way I won't need any other callback classes. That's neat =)
Thank you for your replies!

Alexander.

четверг, 11 февраля 2016 г., 20:22:25 UTC+3 пользователь Matt Martz написал:

[Phil Nelson]

The current thread warns that "only the plugin APIs are guaranteed to work from version to version," and that the API generally "was not intended for direct use."

I've written an extension to the stdout_callback used in roots/trellis to streamline some output and enable text colors in debug msgs.

However, I have decided not to merge the extension because it calls some Ansible modules in a way probably considered to be outside the plugin API. I figure my particular code poses a moderate risk that future Ansible updates could "break" our project for users. My extension's features are merely "nice-to-have" and should only be included if the risk is very low. 

If anyone cares to look, and thinks my code actually poses little risk, I would love to hear it.

Here is the PR and discussion, the code (see most recent 2 commits), and docs describing relevant features.