Make it possible to nest connection plugins

[Filias Heidt]

Hi all,

I'm getting my feet wet in ansible and I'm pretty exited so far. But I think I discovered a use case, which might be of interest for others: 

We have numerous hosts with jails on them and would like to manage the jails with ansible - and not all jails are accessible via SSH, but all hosts are accessible via SSH. I would like to be able to "nest" the jail connection plugin with the ssh connection plugin, so I can connect to a host normally and then connect via the jail plugin to a specific jail and execute commands in there.

Of course I would have to specify on which host to look for the jail.

What do you think about this? If its not too hard to implement, I would consider writing a patch, but would need some help with that.

If thats already possible, please ignore this - I was asking on the IRC-Channel about this and all I found out was, that a jail connection plugin already exists, but is limited to local use. 

Cheerio,

Filias

[Michael DeHaan]

It's not going to be possible to nest them.

What I think you are asking for is some way to do "remote jail management" which is going to be, unfortunately, probably a new connection plugin that might not be easy.

I hate to say it, but this is why cloud VMs and things like bridged networking are so great, where each VM can get an IP.

I'm open to the idea of seeing another connection plugin for this, but I doubt we'd want to maintain it in core.   I could be wrong - I can see the idea of having something like this to be a pretty cool thing for some very specific use cases.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/19385348-1f70-4ac7-ac7a-cdc98abc4de9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Karl E. Jorgensen]

Hi

On Thu, Jul 31, 2014 at 03:38:07AM -0700, Filias Heidt wrote:
> Hi all,
>
> I'm getting my feet wet in ansible and I'm pretty exited so far. But I think I
> discovered a use case, which might be of interest for others:
>
> We have numerous hosts with jails on them and would like to manage the jails
> with ansible - and not all jails are accessible via SSH, but all hosts are
> accessible via SSH.

It sounds more like you need to tweak your SSH config for these jailed
hosts - e.g. a fragment like this in $HOME/.ssh/config:

Host somejailedhost
ProxyCommand ssh jailkeeperhost nc %h %p

This should allow you to use "ssh" (and scp for that matter) to
interact directly with the jailed host. And Ansible should be able to
take advantage of that.

Hope this helps
--
Karl E. Jorgensen

[Filias Heidt]

I thought the more generic "nesting" would be a good thing, because the next guy might want to control Docker instances or Containers on a remote host without them having a public IP. Maybe I'll try to write a connection plugin.

@Karl: This is not possible, because some of the Jails don't even run SSH. You SSH into the Host and run commands inside the jails via "jexec".

[Michael DeHaan]

On Mon, Aug 4, 2014 at 6:14 AM, Filias Heidt <ffh...@gmail.com> wrote:

I thought the more generic "nesting" would be a good thing, because the next guy might want to control Docker instances or Containers on a remote host without them having a public IP. Maybe I'll try to write a connection plugin.

My suspicions are that's going to be technically possible with how connection plugins work (or even in general, very very complicated).  SSH jump hosts would probably be a reasonable thing to do.

 

[Tomasz Kontusz]

On 04.08.2014 12:14, Filias Heidt wrote:

I thought the more generic "nesting" would be a good thing, because the next guy might want to control Docker instances or Containers on a remote host without them having a public IP. Maybe I'll try to write a connection plugin.

The way connection plugins work, it's not going to be possible to nest existing ones.
But what you can do is, create a new API for "connection plugin layers" that can be put on top of an existing "base" connection plugin.
Something like:

class ConnectionLayer(object):
    def __init__(self, target_address):
        """target_address is a part of ansible_ssh_host for this plugin, like jail's path"""

    def wrap_command(self, command):
        """Wraps the command in chroot/jail/etc."""

    def unwrap_result(self, rc, stdout, stderr):
        """Takes whatever the wrapped command returned after being run by the base connection plugin,
        and returns a new tuple of (rc, stdout, stderr)"""

    def wrap_path(self, path):
        """Takes a path inside container/chroot/jail and translates it into host path."""

I'm not really sure if passing continuations or some "lower layers" object wouldn't make this a better API, but I think it touched the most important point.
I would actually like to see something like this, as I've written a OpenVZ-over-SSH connection plugin at work (and I can already give you a hint: don't try to put host address anywhere but ansible_ssh_host, this variable gets special treatment. I've tried to have separate ansible_vzenter_ctid, and in the end had to rewrite that part).

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/df44d2bb-02fd-437d-8343-bfb30b650c3b%40googlegroups.com.

[Michael DeHaan]

Yeah I'm suspecting this will be really hard to debug, if it works at all.

For instance, theoretically, you should be able to nest anything in anything.

So instead of using SSH jumphosts, maybe you SSH into a SSH.   Well, to do that, you have to move a file to a temporary location on an interemediate host.  (This is a bad example, because jumphosts are way better in this case).

I think it only fits for the "remote jail management cloud" case, which, while I'll say it's interesting, I fear it's too far off the beaten path when most people are going to be moving to a world where the virtual nodes have valid IP addresses.

In any case where this is possible, those nodes could just be surfaced by an inventory script that understands what's out there.

I guess what I'm saying is I'm rather skeptical about the idea of having a wrap system, I could see an implementation and be convinced, but I'm also not sure this is important enough to maintain an architectural layer for it.

Sorry, I don't mean to be crushing a cool idea - it's just going to take some selling and I'm not there yet.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53DFDA86.9060205%40gmail.com.

[Alberto Berti]

oh, i forgot the links to the nsenter stuff... here they are:

- http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/
- someone proposed a local docker connection here
https://github.com/ansible/ansible/pull/7971

cheers,

Alberto

[Alberto Berti]

Hi,

sorry for the confusing precendence of my latter message, but seems that
this message had some problems with gmane, so i'm resending it....
>>>>> "Tomasz" == Tomasz Kontusz <tomasz....@gmail.com> writes:

Tomasz> On 04.08.2014 12:14, Filias Heidt wrote:
>> I thought the more generic "nesting" would be a good thing, because
>> the next guy might want to control Docker instances or Containers on
>> a remote host without them having a public IP. Maybe I'll try to
>> write a connection plugin.

Tomasz> The way connection plugins work, it's not going to be possible to nest
Tomasz> existing ones.
Tomasz> But what you can do is, create a new API for "connection plugin
Tomasz> layers" that can be put on top of an existing "base" connection
Tomasz> plugin.
Tomasz> Something like:

Tomasz> class ConnectionLayer(object):
Tomasz> def __init__(self, target_address):
Tomasz> """target_address is a part of ansible_ssh_host for this
Tomasz> plugin, like jail's path"""

Tomasz> def wrap_command(self, command):
Tomasz> """Wraps the command in chroot/jail/etc."""

Tomasz> def unwrap_result(self, rc, stdout, stderr):
Tomasz> """Takes whatever the wrapped command returned after being run
Tomasz> by the base connection plugin,
Tomasz> and returns a new tuple of (rc, stdout, stderr)"""

Tomasz> def wrap_path(self, path):
Tomasz> """Takes a path inside container/chroot/jail and translates it
Tomasz> into host path."""

any update on this? I would like to use nsenter [1]_ to run commands
inside the container (how to transfer files is a detail i have yet to
analyze) to work on docker containers either locally or on a remote
host, and i would really like to contribute to and use a layer like
this. Unfortunately i'm quite new to ansible and not all the details
involving connections are clear to me.

Tomasz> I would actually like to see something like this, as I've written a
Tomasz> OpenVZ-over-SSH connection plugin at work (and I can already give you
Tomasz> a hint: don't try to put host address anywhere but ansible_ssh_host,
Tomasz> this variable gets special treatment. I've tried to have separate
Tomasz> ansible_vzenter_ctid, and in the end had to rewrite that part).

Nice! openvz is the other virtualization system i use :-)

So, where you store the ctid? in to so other host related variable? Can
you say more about having something like a variable ansible_vzenter_ctid
is not feasible?


Cheers,

Alberto

[Tomasz Kontusz]

I'll see what I can do, but I wouldn't hold my breath about opening the
one I wrote at work.
And hey, it's always a good time to rewrite something :-)

>
> Tomasz> I would actually like to see something like this, as I've written a
> Tomasz> OpenVZ-over-SSH connection plugin at work (and I can already give you
> Tomasz> a hint: don't try to put host address anywhere but ansible_ssh_host,
> Tomasz> this variable gets special treatment. I've tried to have separate
> Tomasz> ansible_vzenter_ctid, and in the end had to rewrite that part).
>
> Nice! openvz is the other virtualization system i use :-)
>
> So, where you store the ctid? in to so other host related variable? Can
> you say more about having something like a variable ansible_vzenter_ctid
> is not feasible?

I'm putting it in ansible_ssh_host like this:
some_container ansible_connection=vzenter ansible_ssh_host=<SSH host>|<CTID>

Using variable other than ansible_ssh_host is not feasible, as they are
not provided to connection plugins.
You can extract them from runner (as connection plugins do get access to
runner), but you can't support the delegate_to: '{{ item }}' pattern
this way (as you don't get access to loop items).
> Cheers,
>
> Alberto
>

[Michael DeHaan]

"I'm putting it in ansible_ssh_host like this:
some_container ansible_connection=vzenter ansible_ssh_host=<SSH host>|<CTID>"

Maybe like this:

ansible_ssh_host=hostname;container=CTID

If that could be implemented natively in Ansible's SSH.py, without doing too much damage, I would be SUPER interested.

I'd like to see something somewhat pluggable, and in order to take it, it would need to support Docker initially.

Though I do feel the "no SSH in containers" statement  is a little religious.  They want to encourage automation, but when the automation uses SSH, that's totally cool :)

Cheers,

Alberto

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53E3DA41.7050904%40gmail.com.

[Tomasz Kontusz]

On 07.08.2014 23:35, Michael DeHaan wrote:

"I'm putting it in ansible_ssh_host like this:
some_container ansible_connection=vzenter ansible_ssh_host=<SSH host>|<CTID>"

Maybe like this:

ansible_ssh_host=hostname;container=CTID

I like that syntax, thanks. I'm thinking about doing something like:
ansible_connection=stack ansible_ssh_host=<connection plugin>=<hostname for that plugin>;<connection layer>=<"hostname" for that layer>

If that could be implemented natively in Ansible's SSH.py, without doing too much damage, I would be SUPER interested.

I don't think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core.
That's because I'd like to see the same layers used with local/ssh/accelerate.

I'd like to see something somewhat pluggable, and in order to take it, it would need to support Docker initially.

Right, as Ansible already has a plugin system I think that should be doable :-)
And being pluggable is a big part of it for me - I don't want to have separate plugin for each base plugin/layer combination.

Though I do feel the "no SSH in containers" statement  is a little religious.  They want to encourage automation, but when the automation uses SSH, that's totally cool :)

For me it was either jump hosts or entering containers, and the latter seemed more robust (I can access containers with broken networking :-))

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53E3DA41.7050904%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgwx54dt5OAZOyU4ZG28JeTwgvKw6cffGAUtAWiScbZiuw%40mail.gmail.com.

[Michael DeHaan]

"I don't think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core.
That's because I'd like to see the same layers used with local/ssh/accelerate."

I'm not sure that's a good idea.

(A) all distros are moving to new enough SSH that paramiko is going to be less important, and with no paramiko there will be no need for accelerate

(B) the concept of "nesting" here is probably going to be way more difficult than just doing it for one plugin.

(C) yes, the local case would require ssh, but this seems less important in the long run.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53E3F519.2030501%40gmail.com.

[Tomasz Kontusz]

On 09.08.2014 01:23, Michael DeHaan wrote:

"I don't think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core.

That's because I'd like to see the same layers used with local/ssh/accelerate."

I'm not sure that's a good idea.

(A) all distros are moving to new enough SSH that paramiko is going to be less important, and with no paramiko there will be no need for accelerate

(B) the concept of "nesting" here is probably going to be way more difficult than just doing it for one plugin.

Yes, it is. I've started hacking on it (see https://github.com/ktosiek/ansible-nested-connections. It only supports chroot, but jail shouldn't be hard to add), and I've stumbled upon some problems:
 * to have a sudo layer as robust as sudo functionality in SSH, layers would need some way to get partial output from commands.
    I'm thinking about using generators or something similar, but considering that "yield from" and mixing yield with return were only added somewhere in py3, it might turn out pretty ugly.
 * put/fetch might need some more freedom than just changing the path on managed host, but that shouldn't be hard to add (except maybe for risking a loop between exec_command and put/fetch).

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgxPYib1LYxQbvoz8daoFhB6hWgtuzS_SajAS9q%3Dz0rwhA%40mail.gmail.com.

[Paul Körbitz]

Hi all,

>  Yes, it is. I've started hacking on it (see https://github.com/ktosiek/ansible-nested-connections. It only supports chroot, but jail shouldn't be hard to add), and I've stumbled upon some problems:

I was wondering what the status was on this one? I also need the *configure jails on ssh-accessible jailhost* functionality. If there is no further development I might try to add support for jails for this.

Cheers
Paul

are running
I was wondering what the status was on this one? I also need the *configure jails on ssh-accessible jailhost* functionality (I'm a bit surprised this is not considered a common use case TBH).

[Michael DeHaan]

No plans for this at this time.

I understand the appealing nature of this one, I think it might make sense as a non-core connection plugin perhaps.

But the usage of FreeBSD jails is small now, and I don't think we'd want to continue to maintain this as sort of a "cloud starter kit".

Hope that makes sense.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/2a66d3ef-6259-4360-93d8-7931c15f1fc0%40googlegroups.com.

[Paul Koerbitz]

Dear Michael,

OK, thanks for the heads up. I'll see if I get this working myself then.

cheers
P

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/snM7n1QGoAw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgzRfDLp6BqUZoj7FYmBsHu15JU8AhFeq9DBqqnkCbQ4ow%40mail.gmail.com.