Kubernetes direct management of LXC containers?

[Mark Blackman]

Hi,

I'm just trying work out if Kubernetes *only* works with Docker management of containers or is able to do any form of direct management of LXC containers *without* involvement of the Docker management service.

Regards,
Mark

[Eric Tune]

At the moment it only supports docker + cadvisor.  We've talked about supporting others, but it hasn't quite risen to the level of the other things we are working on at present.

We'd welcome a PR, particularly from someone who has time to maintain the support.  If you are interested, a good start would be filing an issue on github, where interested folks can discuss what work needs to be done.

[Mark Blackman]

Ok, thanks for confirming. I don't have an existing use case, but it struck me that kubernetes might have its own network-oriented management wrapper for LXC or even use LXD.

Is there any conventional wisdom on preferring LXC containers (or not) to the more loosely defined libcontainer containers?

Regards

Mark

--
You received this message because you are subscribed to a topic in the Google Groups "Containers at Google" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-containers/aWZ0U_SIA6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-contain...@googlegroups.com.
To post to this group, send email to google-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-containers.
For more options, visit https://groups.google.com/d/optout.

[Eric Tune]

If you are a user of containers, I think that  right now you use whatever type of container-launcher integrates best with whatever node container manager and orchestration layer you plan to use.

Shameless plug: there is also https://github.com/google/lmctfy containers.​

[Mark Blackman]

On 20 Jan 2015, at 15:13, 'Eric Tune' via Containers at Google <google-c...@googlegroups.com> wrote:

If you are a user of containers, I think that  right now you use whatever type of container-launcher integrates best with whatever node container manager and orchestration layer you plan to use.

Shameless plug: there is also https://github.com/google/lmctfy containers.​

Thanks, the issue for us is that we don’t use containers at all yet and so it’s a wide-open field of choices at each level. I did wonder how mature and general purpose the lmctfy containers were. I’d have assumed, very mature, given that I understand them to be Google’s internal container technology dating from around 2007.

Kubernetes, despite being a Google-driven technology, doesn’t obviously handle lmctfy containers though does it? If we used lmctfy, what would be the right manager and orchestration layer above that?

Cheers,

Mark

[Brendan Burns]

There really isn't one for lmctfy, though it wouldn't be that hard to add it to kubernetes.

To be honest, the truth is (and this is why kubernetes uses Docker, instead of lmctfy) that the Docker folks did a really great job of creating an ecosystem of easy to use tooling around the Docker daemon and container image format.

So much so, that it is the primary use case for kubernetes.

That said, we have always intended for it to be container technology neutral, and we'd gladly accept PRs to handle other container formats in the kubelet.  We have some larger things to get settled as we drive to 1.0, but after we get there, we'll also look into supporting other formats ourselves.

--brendan

--
You received this message because you are subscribed to the Google Groups "Containers at Google" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-contain...@googlegroups.com.

[Tim Hockin]

What Brendan said. If someone wanted to make LXC or LMCTFY*
containers work, I'd be happy to consult on it.

* LMCTFY is not a fully-formed solution as it stands. It is all about
isolation and nothing about images or filesystem layout.

On Tue, Jan 20, 2015 at 11:16 PM, 'Brendan Burns' via Containers at

[Mark Blackman]

Thanks for all the pointers, guys. Very helpful for surveying the landscape. I'll assume you mean problem report by PR rather than pull request and see about filing one to look for lmctfy and LXC support in kubernetes.  It seems like the best near-term option is to use Docker as the direct container management technology, with kubernetes handling the multi-host management.

Regards 

Mark

You received this message because you are subscribed to a topic in the Google Groups "Containers at Google" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-containers/aWZ0U_SIA6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-contain...@googlegroups.com.

[Tim Hockin]

No, I think we meant a Pull Request :). There is SO MUCH to do, we will not get to LXC or LMCTFY any time soon.  If you want it, the best way to get it is to help.  If that's not possible, it's OK too, it will just take longer.

Tim

[Daniel Middleton]

LXC/LXD support in Kubernetes would be awesome. Forming containers around a single process (hey Docker) is obviously the ideal in terms of a micro-service infrastructure, but a lot of us don't yet have this pleasure (e.g, those of us tasked with moving an organisation in to 'DevOps' but with legacy services to support in the meantime). This is obviously not trivial to implement, but it shouldn't be the goliath task it was pre-LXD now that we have a means of nicely transferring and migrating LXC containers around hosts, given, we're early days. I've been looking at Mesos/Marathon, CoreOS, etc, etc, but nothing seems to solve the problem of actually orchestrating a cluster of non-Docker containers. I'd also be willing to consult/contribute if someone would like to make that 'first step'.

[Tim Hockin]

The first step is the runtime abstraction being worked on right now to support rkt. Take a look and see what parts of your use case are not covered.

On May 10, 2015 10:45 AM, "Daniel Middleton" <dannyd...@gmail.com> wrote:

LXC/LXD support in Kubernetes would be awesome. Forming containers around a single process (hey Docker) is obviously the ideal in terms of a micro-service infrastructure, but a lot of us don't yet have this pleasure (e.g, those of us tasked with moving an organisation in to 'DevOps' but with legacy services to support in the meantime). This is obviously not trivial to implement, but it shouldn't be the goliath task it was pre-LXD now that we have a means of nicely transferring and migrating LXC containers around hosts, given, we're early days. I've been looking at Mesos/Marathon, CoreOS, etc, etc, but nothing seems to solve the problem of actually orchestrating a cluster of non-Docker containers. I'd also be willing to consult/contribute if someone would like to make that 'first step'.

[Daniel Middleton]

Thanks, I admittedly completely overlooked rkt, most of my efforts have been LXC based up to now, I'll need to read further in to how rkt differs from LXC. For anyone else interested, work on the runtime abstraction being worked on, there's a thread here:  https://github.com/GoogleCloudPlatform/kubernetes/issues/4404

You received this message because you are subscribed to a topic in the Google Groups "Containers at Google" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-containers/aWZ0U_SIA6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-contain...@googlegroups.com.

[Brendan Burns]

It is also mostly implemented and merged in the codebase @ head, if you'd prefer to browse source.

Brendan

[Daniel Middleton]

FYI, the runtime abstraction has been completed and the task closed 4 hours ago, see: https://github.com/GoogleCloudPlatform/kubernetes/issues/4404

So, would anyone care to advise as to how we'd get started with LXD?

[Victor Marmol]

I am not super familiar with LXD, but from what I've seen it is similar to Docker in API and how you interact with it (as in: declarative and no pod abstraction). Looking at the Docker runtime would probably be a good place to start. See how the different runtime calls are handled and try to do the same in an LXD-way. The biggest thing to try to replicate initially will be the pod abstraction itself.

[Ranjib Dey]

Hi

I am reviving this thread,

I am interested in having a LXC (may be LXD as well) driver on k8s. Primarily to run full blown systems (with init/cron etc) as well as having user namespace / unprivileged containers. I already maintain a LXC & systemd driver for nomad(another scheduler) but i want to try out k8s as well. I am familiar with the lxc/lxd ecosystem as well as go-lxc binding and I'll start working on k8s-lxc integration as soon as I am familiar with the k8s internal abstractions. Currently I'm going through the codebase, and any help on this will help a lot (like is there a driver/executor interface I can refer to, etc).

Also ping me if you want to collaborate, or if anyone is actively working on this, I would love to work on this, or test this out.

cheers

ranjib

On Sunday, January 18, 2015 at 2:23:05 PM UTC-8, Mark Blackman wrote:

[Tim Hockin]

I think we're not at all against this, but the "runtime" abstraction
is undergoing a lot of change, so it might be a challenge right now
(or it might be an opportunity to contribute :)

> --
> You received this message because you are subscribed to the Google Groups
> "Containers at Google" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-contain...@googlegroups.com.
> To post to this group, send email to google-c...@googlegroups.com.

> Visit this group at https://groups.google.com/group/google-containers.

[Vishnu Kannan]

We are updating the abstractions to make integration easier. I'd be happy to assist you if you want to get started right away.

[mal...@gmail.com]

How is now the "runtime" abstraction is it stable?

[matt.bar...@gmail.com]

How's the progress on this? Are there any examples of a working code base anywhere or a tutorial on how to get started?