OpenCompose - A higher level abstraction to define applications for Kubernetes

[Pradeepto Bhattacharya]

Hi,

At KubeCon Dev Summit, we had an excellent conversation around "Application/Service Definition" - a higher level abstraction for Kubernetes resources. Talking to many Kubernetes contributors during the conference and dev summit, I felt that there is a definite need for such a higher level declarative abstraction (and I felt that we got some validation about the idea).

Since then, we (few of us here at Red Hat) have been slowly but surely working towards it. We call it "OpenCompose" [1] (obviously, we aren't very creative with naming things).  Recently, we saw a presentation from Brian Grant at sig-apps meeting [2]. The proposal document [3] is very detailed (thanks Brian) and a great read. We found that there are a bunch of overlapping ideas.

At this point, OpenCompose is a tool that understands a simple YAML syntax (the OpenCompose file reference). A micro service can be defined using this syntax and the OpenCompose tool can generate Kubernetes configurations from it. We have few examples and some documentation. We are working towards augmenting the file reference / syntax.

The main goal for OpenCompose is to be easy to use application/microservice definition that developers can use without learning much of Kubernetes concepts. It should be very easy to write a simple application definition and from there on the tooling takes over. How easy? Just think about UX of Docker Compose and how easy it is for developers to write a compose file and bring up application. We have been extensively contributing to Kompose because we strongly believe that it helps developers migrate from Docker Compose to Kubernetes. But not all concepts of Kubernetes can be mapped 1-1 to Docker Compose and vice versa and this made us think about the problem. We have factored some of our thoughts into current implementation of OpenCompose file reference [4] and tool.

There are many features from Brian's document that we would like to have in OpenCompose. And I think, it can be guiding document for OpenCompose although Brian's document is about common-of-the-shelf (COTS), I am sure that we can leverage from it.

We would love to present the current version of OpenCompose at the next available slot during sig-apps weekly meeting.

There is definitely a huge scope to improve and we need feedback from the community. What does the sig-apps folks think about what we have done until now? How can we collaborate and work together on this idea?

If you have questions about OpenCompose, please reply to this thread , a bunch of us are subscribed to this list and we will get back to you.

[1] https://github.com/redhat-developer/opencompose

[2] followup on declarative application management from today's SIG meeting

[3] Proposal document

[4] OpenCompose File Reference

Thanks,

Pradeepto (on behalf of my teammates)

--

Pradeepto Bhattacharya

[Alex Clemmer]

I think this is great, thanks for sharing.

It seems to me that tools like this, which lower the skill floor
required to get started, have the potential to offer a lot of value to
the ecosystem. It seems to me (and I am an outsider, so this might be
wrong) that most tools that make this attempt do so by offering a new
API that maps to the k8s API objects, and I'm curious to hear why you
chose this route instead of investing in tooling that makes it much
easier to interact with the core k8s API objects themselves?

> --
> You received this message because you are subscribed to the Google Groups
> "kubernetes-sig-apps" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-sig-...@googlegroups.com.
> To post to this group, send email to kubernete...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-sig-apps/CADe3W8WaxRS66hjjoDQ7ySnQnTGxWnHbjc%2B3184uCycc-5hJKg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.



--

__
Transcribed by my voice-enabled refrigerator, please pardon chilly
messages.

[pska...@mirantis.com]

Looking on this description - what advantages this have to https://github.com/kelseyhightower/compose2kube

or even to already existing in k8s umbrela https://github.com/kubernetes-incubator/kompose ?

Looks like opencompose introduces new (possibly better) interface. But still new to learn by potential users of existing solutions.

[Tomas Kral]

Hi,

As I'm working on both projects (Kompose and OpenCompose), I can give you quick comparison and also reason why we started OpenCompose.

There are similar ideas behind both projects, but one big difference is file format.

Kompose is using already existing docker-compose definition. docker-compose yaml file is great simple and easy to learn. Most of the stuff from docker-compose file can be easily mapped to Kubernetes, but there are some problems.

Big benefit there is that you can take existing stuff and use it on Kubernetes, but if you want to benefit from Kubernetes specific features (for example Pods), you can't :-( Docker-compose format was designed with slightly different view on container world that Kubernetes uses.

We started OpenCompose because we think that there is need for something that is native to Kubernetes and easy to learn as docker-compose.

Yes you are right, it is still something new to learn for potential users. But OpenCompose has one big benefit to Kompose, it allows you to  leverage Kubernetes feature that you can't with docker-compose format.

I'm happy to discuss this more. 

We are open to any ideas, feedback, anything ;-)

--

Tomas

--

You received this message because you are subscribed to the Google Groups "kubernetes-sig-apps" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-...@googlegroups.com.
To post to this group, send email to kubernete...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-apps/eddf51e8-2cf0-4454-b7f3-257257ce35d2%40googlegroups.com.

[Brian Grant]

On Fri, Mar 3, 2017 at 3:43 AM, Tomas Kral <tk...@redhat.com> wrote:

Hi,

As I'm working on both projects (Kompose and OpenCompose), I can give you quick comparison and also reason why we started OpenCompose.

There are similar ideas behind both projects, but one big difference is file format.

Kompose is using already existing docker-compose definition. docker-compose yaml file is great simple and easy to learn. Most of the stuff from docker-compose file can be easily mapped to Kubernetes, but there are some problems.

Big benefit there is that you can take existing stuff and use it on Kubernetes, but if you want to benefit from Kubernetes specific features (for example Pods), you can't :-( Docker-compose format was designed with slightly different view on container world that Kubernetes uses.

We started OpenCompose because we think that there is need for something that is native to Kubernetes and easy to learn as docker-compose.

Yes you are right, it is still something new to learn for potential users. But OpenCompose has one big benefit to Kompose, it allows you to  leverage Kubernetes feature that you can't with docker-compose format.

I'm happy to discuss this more. 

We are open to any ideas, feedback, anything ;-)

Thanks. I'll try to find time to take a look.

For potential historical interest, I wrote a simple Fig-like tool back in 2014:

https://github.com/bgrant0607/kubernetes/blob/b8959bf146a8d7bbbd2bbd7fa676f2e3fe711c3a/contrib/srvexpand/README.md

 

--

Tomas

On Fri, Mar 3, 2017 at 12:04 PM <pska...@mirantis.com> wrote:

Looking on this description - what advantages this have to https://github.com/kelseyhightower/compose2kube

or even to already existing in k8s umbrela https://github.com/kubernetes-incubator/kompose ?

Looks like opencompose introduces new (possibly better) interface. But still new to learn by potential users of existing solutions.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-apps" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-apps+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-apps@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-apps/eddf51e8-2cf0-4454-b7f3-257257ce35d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "kubernetes-sig-apps" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-apps+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-apps@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-apps/CAAYTw5NZgjjX350q3q2W_-WweoBNAsCKHuhAUGhBPCOm7%3DZ1Xw%40mail.gmail.com.

[Sebastien Goasguen]

> On Mar 2, 2017, at 8:06 PM, Alex Clemmer <al...@heptio.com> wrote:
>
> I think this is great, thanks for sharing.
>
> It seems to me that tools like this, which lower the skill floor
> required to get started, have the potential to offer a lot of value to
> the ecosystem. It seems to me (and I am an outsider, so this might be
> wrong) that most tools that make this attempt do so by offering a new
> API that maps to the k8s API objects, and I'm curious to hear why you
> chose this route instead of investing in tooling that makes it much
> easier to interact with the core k8s API objects themselves?
>

I can’t speak for Pradeepto, but since I started kompose I can speak of the driving reason to do it.

Docker was being adopted by developers heavily, quickly developers realized they needed multi-container and to avoid writing massive bash script that contained `docker run` command, fig and then docker-compose came into play. So developers ended up with docker-compose files.

Now we (skippbox) believed Kubernetes would be the production platform and that ops would deploy containerized apps on k8s. So developers would give a docker-compose file to ops guys, or folks would need to learn k8s quickly.

kompose was/is an on-boarding tool that brings docker-compose users onto k8s quickly. Once you get the k8s primitives, you leave docker-compose behind.

The mapping will never be perfect, but it gets you 80% of the way there quickly and you can then spend time learning the k8s primitives.

-sebastien

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-apps/CAKMv2xzLcNRVKh6qzA2woMPq_6Q%3DgsSTPR-bGyksBkqAwLHNdA%40mail.gmail.com.

[Brian Grant]

+SIG CLI

On Wed, Mar 1, 2017 at 3:26 AM, Pradeepto Bhattacharya <prad...@redhat.com> wrote:

Hi,

At KubeCon Dev Summit, we had an excellent conversation around "Application/Service Definition" - a higher level abstraction for Kubernetes resources. Talking to many Kubernetes contributors during the conference and dev summit, I felt that there is a definite need for such a higher level declarative abstraction (and I felt that we got some validation about the idea).

Since then, we (few of us here at Red Hat) have been slowly but surely working towards it. We call it "OpenCompose" [1] (obviously, we aren't very creative with naming things).  Recently, we saw a presentation from Brian Grant at sig-apps meeting

Presentation:

https://docs.google.com/presentation/d/17AqTn5QisaD2F3pz2buZUVW1fdlIW-Mv_zIiRBrTaSY/edit#slide=id.p

 

[2]. The proposal document [3] is very detailed (thanks Brian) and a great read. We found that there are a bunch of overlapping ideas.

At this point, OpenCompose is a tool that understands a simple YAML syntax (the OpenCompose file reference). A micro service can be defined using this syntax and the OpenCompose tool can generate Kubernetes configurations from it. We have few examples and some documentation. We are working towards augmenting the file reference / syntax.

The main goal for OpenCompose is to be easy to use application/microservice definition that developers can use without learning much of Kubernetes concepts. It should be very easy to write a simple application definition and from there on the tooling takes over. How easy? Just think about UX of Docker Compose and how easy it is for developers to write a compose file and bring up application. We have been extensively contributing to Kompose because we strongly believe that it helps developers migrate from Docker Compose to Kubernetes. But not all concepts of Kubernetes can be mapped 1-1 to Docker Compose and vice versa and this made us think about the problem. We have factored some of our thoughts into current implementation of OpenCompose file reference [4] and tool.

There are many features from Brian's document that we would like to have in OpenCompose. And I think, it can be guiding document for OpenCompose although Brian's document is about common-of-the-shelf (COTS), I am sure that we can leverage from it.

We would love to present the current version of OpenCompose at the next available slot during sig-apps weekly meeting.

There is definitely a huge scope to improve and we need feedback from the community. What does the sig-apps folks think about what we have done until now? How can we collaborate and work together on this idea?

If you have questions about OpenCompose, please reply to this thread , a bunch of us are subscribed to this list and we will get back to you.

Good start!

Things I'd like to see investigated/addressed:

• Handle multiple API versions
• Declarative specification of metadata across multiple resources (esp. API versions, labels, and annotations), to reduce boilerplate
• Apply commonly recommended labels
• Represent more controller types (e.g., StatefulSet)
• Perhaps support multiple controllers per service, such as for canary deployments
• Ensure most/all of the spec automatically tracks changes to the K8s API
  
• Handle extension APIs from aggregated API servers and ThirdPartyResource
• Automatically distinguish these configs from K8s resource manifests in a robust manner
  
• Consider a syntax that's somewhat less fragile than yaml (HCL? concrete subset of jsonnet?)
• Declarative specification of common generator commands, such as create secret and create configmap. Once we move away from strict API resources, we have the freedom to introduce client-specific syntax for this type of thing.
• Ensure it can work with apply
• Autogenerate reference documentation

 

[1] https://github.com/redhat-developer/opencompose

[2] followup on declarative application management from today's SIG meeting

[3] Proposal document

[4] OpenCompose File Reference

Thanks,

Pradeepto (on behalf of my teammates)

--

Pradeepto Bhattacharya

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-apps" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-apps+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-apps@googlegroups.com.

[adam....@container-solutions.com]

Hi Pradeepto, thanks for sharing, I really like where this project is going.

I think the goal of helping beginners get started is a good one but too limited. In my opinion this kind of higher level abstraction is important not only for beginners but to anyone who doesn't do fancy stuff like trying to run non-12-factor legacy apps. At the client I'm working with as a consultant right now we have a set of microservices we are running on Kubernetes. It's two applications with 6 services in total and we are already getting close to manifest file hell. To run one microservice we need 4 manifest files (deployment, service, hpa, ingress). I think a higher level of abstraction could help this problem by mapping to reasonable defaults in the actual Kubernetes API objects - at the moment this is theory I would have to look into how much could be reasonably abstracted away.

Another area where having a single service definition instead of a bunch of loosely coupled object would be avoiding mistakes. The above mentioned 4 objects are loosely coupled using labels. But in an ideal use-case of running 12-factor microservices I would like that linking to be done without me having to type the name of the service in 4-5 different places. 

I could also imagine a runtime server-side component which would verify that all object got created successfully. Because of the loose coupling it's hard to tell whether your service is running fine because there all 4 objects that can fail independently or just be misconfigured and not pointing to each other (e.g. typo in Service selector).

I might be projecting my own use-case into a project that has different goals, but maybe these goals could be covered by OpenCompose alongside the goal of helping new users get up to speed. What do you think?

Adam

[Brian Grant]

On Sat, Mar 4, 2017 at 3:21 AM, adam.sandor via kubernetes-sig-apps <kubernete...@googlegroups.com> wrote:

Hi Pradeepto, thanks for sharing, I really like where this project is going.

I think the goal of helping beginners get started is a good one but too limited. In my opinion this kind of higher level abstraction is important not only for beginners but to anyone who doesn't do fancy stuff like trying to run non-12-factor legacy apps.

If you're just running 12-factor apps, have you considered one of the multitude of PaaSes on K8s, such as Openshift, Deis Workflow, Rancher, Kel, GiantSwarm, Hasura, ...? Or if you just need functions, Fission, Kubeless, Iron.io, ...?

What features is kubectl run lacking that you would need?

 

At the client I'm working with as a consultant right now we have a set of microservices we are running on Kubernetes. It's two applications with 6 services in total and we are already getting close to manifest file hell. To run one microservice we need 4 manifest files (deployment, service, hpa, ingress).

Is the number of files a problem? You can put multiple resources into a single file.

https://kubernetes.io/docs/user-guide/managing-deployments/#organizing-resource-configurations

 

I think a higher level of abstraction could help this problem by mapping to reasonable defaults in the actual Kubernetes API objects - at the moment this is theory I would have to look into how much could be reasonably abstracted away.

Another area where having a single service definition instead of a bunch of loosely coupled object would be avoiding mistakes. The above mentioned 4 objects are loosely coupled using labels. But in an ideal use-case of running 12-factor microservices I would like that linking to be done without me having to type the name of the service in 4-5 different places. 

Would a template work for your use cases? 

I could also imagine a runtime server-side component which would verify that all object got created successfully. Because of the loose coupling it's hard to tell whether your service is running fine because there all 4 objects that can fail independently or just be misconfigured and not pointing to each other (e.g. typo in Service selector).

I might be projecting my own use-case into a project that has different goals, but maybe these goals could be covered by OpenCompose alongside the goal of helping new users get up to speed. What do you think?

Adam

On Wednesday, March 1, 2017 at 12:27:06 PM UTC+1, Pradeepto Bhattacharya wrote:

Hi,

At KubeCon Dev Summit, we had an excellent conversation around "Application/Service Definition" - a higher level abstraction for Kubernetes resources. Talking to many Kubernetes contributors during the conference and dev summit, I felt that there is a definite need for such a higher level declarative abstraction (and I felt that we got some validation about the idea).

Since then, we (few of us here at Red Hat) have been slowly but surely working towards it. We call it "OpenCompose" [1] (obviously, we aren't very creative with naming things).  Recently, we saw a presentation from Brian Grant at sig-apps meeting [2]. The proposal document [3] is very detailed (thanks Brian) and a great read. We found that there are a bunch of overlapping ideas.

At this point, OpenCompose is a tool that understands a simple YAML syntax (the OpenCompose file reference). A micro service can be defined using this syntax and the OpenCompose tool can generate Kubernetes configurations from it. We have few examples and some documentation. We are working towards augmenting the file reference / syntax.

The main goal for OpenCompose is to be easy to use application/microservice definition that developers can use without learning much of Kubernetes concepts. It should be very easy to write a simple application definition and from there on the tooling takes over. How easy? Just think about UX of Docker Compose and how easy it is for developers to write a compose file and bring up application. We have been extensively contributing to Kompose because we strongly believe that it helps developers migrate from Docker Compose to Kubernetes. But not all concepts of Kubernetes can be mapped 1-1 to Docker Compose and vice versa and this made us think about the problem. We have factored some of our thoughts into current implementation of OpenCompose file reference [4] and tool.

There are many features from Brian's document that we would like to have in OpenCompose. And I think, it can be guiding document for OpenCompose although Brian's document is about common-of-the-shelf (COTS), I am sure that we can leverage from it.

We would love to present the current version of OpenCompose at the next available slot during sig-apps weekly meeting.

There is definitely a huge scope to improve and we need feedback from the community. What does the sig-apps folks think about what we have done until now? How can we collaborate and work together on this idea?

If you have questions about OpenCompose, please reply to this thread , a bunch of us are subscribed to this list and we will get back to you.

[1] https://github.com/redhat-developer/opencompose

[2] followup on declarative application management from today's SIG meeting

[3] Proposal document

[4] OpenCompose File Reference

Thanks,

Pradeepto (on behalf of my teammates)

--

Pradeepto Bhattacharya

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-apps" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-apps+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-apps@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-apps/1e6e1a43-05ce-4546-8748-111e9e217aa3%40googlegroups.com.

[adam....@container-solutions.com]

Thanks for your reply Brian, I will reply point by point. In general though you're right that we might not be needing any radical new level of abstraction - maybe a simple templating solution would be enough. I think I'll open a new thread about this.

If you're just running 12-factor apps, have you considered one of the multitude of PaaSes on K8s, such as Openshift, Deis Workflow, Rancher, Kel, GiantSwarm, Hasura, ...? Or if you just need functions, Fission, Kubeless, Iron.io, ...?

That's a good point. I agree what we need is a kind of a PaaS but the solutions that are out there don't really suit us, and we can't go with functions. But it was a good push for me to go and check each of these out. I feel none of them provide what I'm looking for.

• Openshift: It's a distribution of Kubernetes therefore I cannot install it on GKE. Doesn't really provide different abstraction from pure K8s anyway.
  
• Deis Workflow: Looks cool on first sight however after reading through the docs it seems to me all the features are just thin wrappers around existing K8s functionality. I don't know if I missed some point there but I don't see any advantage there (except the builder component, but we have that covered by Codeship anyway)
  
• Rancher: It's more a K8s installer then a PaaS, unless I missed something.
  
• Kel: Might be good, but doesn't have enough documentation to find out what it's doing.
  
• GiantSwarm: We already opted for GKE and they are the competition
• Hasura: Doesn't seem to offer any high level application abstractions. 
  

Is the number of files a problem? You can put multiple resources into a single file.

It's not the number of files, rather the fact that the objects are loosely coupled. I also was feeling we are repeating ourselves but taking another look there isn't much repetition in those files.

Would a template work for your use cases? 

I guess that might solve the problem too.

To post to this group, send email to kubernete...@googlegroups.com.