Should pipelines & metadata form the foundation of KF?

472 views
Skip to first unread message

Jeremy Lewi

unread,
Oct 7, 2020, 12:17:32 PM10/7/20
to kubeflow-discuss
Hi Folks,

Over the past couple of months we've been working hard to divide Kubeflow into working groups responsible for different KF applications. 

We now have 5 different work groups (listed here) that own all the major applications.

There is an open question about what to do about some of the remaining applications/pieces that don't fit neatly into these work groups. These include

Control Plane/kfctl/manifests - (https://github.com/kubeflow/community/pull/402)
Auth/Dex/Profile Controller
Reference Architectures / generic KFDefs

We've been debating what to do at the recent community meetings. I wanted to start a thread to give everyone an opportunity to chime in.

To summarize it looks like there are two proposals on the table

1. Spin up appropriate WGs to own and maintain these applications going forward
2. KF stops assuming responsibility for bundling and deploying all the applications
  • Each application should be installable on its own with that responsibility assumed by the applications owners and/or WG
  • Vendors would be responsible for composing the applications into distributions and providing glue (e.g. Dex or middleware like ISTIO) at their discretion.
#2 raises the question what ties the various applications into a cohesive platform?

One possibility is that pipelines & metadata not deployment is what binds together the applications into a cohesive platform.

This idea is something we first discussed at our 2019 summit. All Kubeflow applications would follow some conventions that enable:
  1. using them in pipelines
  2. consistently logging metadata
Adhering to these conventions is what would distinguish a KF application from a non KF application.  

Thoughts?
J





David Aronchick

unread,
Oct 7, 2020, 2:24:10 PM10/7/20
to Jeremy Lewi, kubeflow-discuss
I think this is a really interesting thought and I like this direction.

I wonder if there's one more component that we need as part of KF and that's the installation contracts/tools/etc. Would that be part of "what kubeflow is"?

Also, there's something to be said for testing infrastructure to ensure that people who want to contribute components know HOW to be well supported and/or keep up to date with new versions. Would that also be something KF would provide?

Thanks!

--
You received this message because you are subscribed to the Google Groups "kubeflow-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com.

Constantinos Venetsanopoulos

unread,
Oct 7, 2020, 3:22:01 PM10/7/20
to David Aronchick, Jeremy Lewi, kubeflow-discuss
Hello all,

Being part of yesterday's community meeting my understanding is that the
community is hungry to see Kubeflow succeed, but this doesn't suffice on
its own. At the end of the day everything goes down to people putting
very specific effort.

We understand that the focus of almost every vendor has currently
shifted to specific components of Kubeflow, where everyone sees
different value for different reasons, and away from looking at Kubeflow
as a platform.

Our premise at Arrikto, and the reason we got involved in Kubeflow in
the first place back in 2018, is that for Kubeflow to be successful and
become the de facto ML platform on Kubernetes, it needs to be considered
a coherent platform with simple, portable, and scalable end-to-end
workflows. And this is how it should be advertised externally to end users.

For this to happen, someone needs to take responsibility and most
importantly put the really significant, hard engineering work to
maintain some of the components that Jeremy listed above, which tie
everything together. To have the bare minimum base in place, so that
vendors can then build on top or mix and match apps.

Since no one seems to be willing to step up, and most of these
components will essentially become obsolete and abandoned going forward,
we decided to bite the bullet and step up to do the actual work.

We have been heavy contributors all the past years to almost all of
them, so our plan is to propose and lead a WG that will incorporate the
things we think make sense to keep maintaining, and not get abandoned.

Driving from Jeremy's and David's proposal, we would love to work very
closely with the KFP team to ensure a seamless integration, as happened
to be the case in the past.

Thank you and looking forward to realizing the original vision of Kubeflow.

Best,
Constantinos

On 07/10/20 11:23, David Aronchick wrote:
> I think this is a really interesting thought and I like this direction.
>
> I wonder if there's one more component that we need as part of KF and
> that's the installation contracts/tools/etc. Would that be part of
> "what kubeflow is"?
>
> Also, there's something to be said for testing infrastructure to
> ensure that people who want to contribute components know HOW to be
> well supported and/or keep up to date with new versions. Would that
> also be something KF would provide?
>
> Thanks!
>
> On Wed, Oct 7, 2020 at 9:17 AM Jeremy Lewi <jer...@lewi.us
> <mailto:jer...@lewi.us>> wrote:
>
> Hi Folks,
>
> Over the past couple of months we've been working hard to divide
> Kubeflow into working groups responsible for different KF
> applications. 
>
> We now have 5 different work groups (listed here
> <https://github.com/kubeflow/community/blob/master/wg-list.md>)
> that own all the major applications.
>
> There is an open question about what to do about some of the
> remaining applications/pieces that don't fit neatly into these
> work groups. These include
>
> central dashboard -
> (https://github.com/kubeflow/community/issues/380
> <https://github.com/kubeflow/community/issues/380>)
> pod defaults controller -
> (https://github.com/kubeflow/community/issues/381
> <https://github.com/kubeflow/community/issues/381>)
> Control Plane/kfctl/manifests -
> (https://github.com/kubeflow/community/pull/402
> <https://github.com/kubeflow/community/pull/402>)
> Auth/Dex/Profile Controller
> Reference Architectures / generic KFDefs
>
> We've been debating what to do at the recent community meetings. I
> wanted to start a thread to give everyone an opportunity to chime in.
>
> To summarize it looks like there are two proposals on the table
>
> 1. Spin up appropriate WGs to own and maintain these applications
> going forward
> 2. KF stops assuming responsibility for bundling and deploying all
> the applications
>
> * Each application should be installable on its own with that
> responsibility assumed by the applications owners and/or WG
> * Vendors would be responsible for composing the applications
> into distributions and providing glue (e.g. Dex or middleware
> like ISTIO) at their discretion.
>
> #2 raises the question what ties the various applications into a
> cohesive platform?
>
> One possibility is that pipelines & metadata not deployment is
> what binds together the applications into a cohesive platform.
>
> This idea is something we first discussed at our 2019 summit
> <https://docs.google.com/presentation/d/13a3shc98F-G779tnNFXb4ZYfhniInI--NtUanIivkOM/edit#slide=id.g51ce843281_2_1>.
> All Kubeflow applications would follow some conventions that enable:
>
> 1. using them in pipelines
> 2. consistently logging metadata
>
> Adhering to these conventions is what would distinguish a KF
> application from a non KF application.  
>
> Thoughts?
> J
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

David Aronchick

unread,
Oct 7, 2020, 3:31:51 PM10/7/20
to Constantinos Venetsanopoulos, Jeremy Lewi, kubeflow-discuss
We really really appreciate your work! I guess my question is, if the company/contributor of the component DOESN'T step up, shouldn't the component be removed over time? Or put into an "unsupported" state?

E.g. if the contributors of Katib didn't do any work, and no one was willing to support it, then shouldn't it (rightfully) die? (I'm not saying they're saying this, I'm just using them as an example!)

My major concern is if someone comes along and contributes something, but then gives up on it, why would the community take it forward if no one cared?

I love the idea of a "Core User Journey Working Group" to guide & own this - thinking about the medium post which made it around the web recently, a team as dedicated as yours to own this would be FANTASTIC.

Constantinos Venetsanopoulos

unread,
Oct 7, 2020, 3:43:39 PM10/7/20
to David Aronchick, Jeremy Lewi, kubeflow-discuss
Hello David,

Thank you for your support on this!

We totally agree here. If someone contributes a component like KFP, or
KFServing, or Katib, or Notebooks, and then stops supporting it, it
should rightfully die.

But the reason for this WG is not supporting a specific user facing
component like the above. It is for supporting the underlying plumbing
that brings everything together. Traditionally, this hard, technical
work was mostly done by Google engineers, and others contributing
alongside, including us. This doesn't seem to be the case anymore and
this is what causes all this confusion to people. Our view is that this
the answer to "what is Kubeflow" that everyone is asking for the last
3-4 months in every community meeting. This is why I also mentioned,
that most probably we won't include everything that's left out in this
WG, we will pick the ones that we think is the bare minimum needed, and
some will eventually die.

And yes, on the end user facing side, we want to enable simple CUJs that
span Kubeflow components and showcase the power of an e2e workflow and
of a coherent platform.

Best,
Constantinos
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >     <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
>  <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> > <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>

Michael Tanenbaum

unread,
Oct 7, 2020, 4:45:09 PM10/7/20
to Constantinos Venetsanopoulos, David Aronchick, Jeremy Lewi, kubeflow-discuss
Hi all -

I would like to second the sentiments expressed by Constantinos.

As a vendor currently selling an enterprise distro, we stand in support of Kubeflow because it shows the greatest potential to be "a coherent platform with simple, portable, and scalable end-to-end workflows."

We are working to align our senior leadership on the benefits for a greater commitment to assisting and developing against upstream goals, in addition to the work we do on our own distro. The cohesiveness of KF, its position as the de facto standard for AI/ML on Kubernetes, as well as the strength of the community and vendor backing is critically interfaced with our success internally and externally with current and future customers.

FWIW, we would very much like to see KF adhere to the vision as articulated by Constantinos. Please let me know if there are any immediate areas of need where I can invest my time and bring along others from our team. We look forward to engaging.

All the best,

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubeflow-discuss/63dde059-37b6-507d-aeef-6cd8aa13426d%40arrikto.com.


--
Michael Tanenbaum
Principal Sales Engineer, D2iQ

Animesh Singh

unread,
Oct 7, 2020, 5:03:50 PM10/7/20
to Michael Tanenbaum, Constantinos Venetsanopoulos, David Aronchick, Jeremy Lewi, kubeflow-discuss
Thanks all. Echoes some of the points raised in the community call yesterday. Few other thoughts in my head

1. The general philosophy of doing fewer things, and doing them well should drive the vision of Kubeflow moving forward.
2. In the past we have been more generous of spawning projects in Kubeflow umbrella, and we should now put higher bar and guidelines with a year roadmap before something becomes part of Kubeflow. And also be ok with "retiring" projects if they are not being invested in.
3. Now coming to the "end-to-end" story. As is evident from the thread, there are two kinds of consumers - folks who are more interested in one or couple of the verticals (pipelines, serving, training etc. ) or folks who look at Kubeflow as a single horizontal entity, and distribute as such. There is room for both. While we have been succeeding with WGs on the "vertical" front, the "horizontal" story is sort of falling apart. We don't see representatives from respective organizations on community calls who would be interested in carrying that mantle forward.
4. The best way to tackle this can be to find consumers and users who benefit by positioning Kubeflow as a distro, and by virtue of it are the ones they carry the mantle forward on that story. On the other hand, folks who are more interested in verticals and consuming/redistributing them, should carry the burden there. 
5. In lieu of not getting "owners" who are invested in the horizontal story, its best to start communicating and setting the right expectations to the end user community. by positioning some parts of Kubeflow as "reference architectures", with "sample" implementations, instead of "solid supported implementations"

On #5 above, if we do focus the story on vertical more, as opposed to horizontal, agree with Jeremy's proposal to set some minimum criteria for Kubeflow applications.

1. Every component/application to provide and support a KFP component which can be used in Pipelines. This will be a low-touch cohesive way for applications to opt-in the "Pipeline Train". Creating a component for KFP is well scoped and nicely defined.
2. If we can define clarity around how to plug in "Metadata", for each component in a standard way, this can also be made a minimum.
3. in addition to each WG providing a Kustomize configuration for deployment of the app from the WG 

Beyond that, if we do find a solid work groups owning a central dashboard, auth/authz, ingress etc, end to end installer etc, rest of the criteria can be laid out in an "optional" fashion.

Thanks,

Animesh


Constantinos Venetsanopoulos

unread,
Oct 7, 2020, 6:50:25 PM10/7/20
to Animesh Singh, Michael Tanenbaum, David Aronchick, Jeremy Lewi, kubeflow-discuss
Hello Animesh,

Thank you very much for your feedback. Please find comments inline:

On 07/10/20 14:03, Animesh Singh wrote:
> Thanks all. Echoes some of the points raised in the community call
> yesterday. Few other thoughts in my head
>
> 1. The general philosophy of doing fewer things, and doing them well
> should drive the vision of Kubeflow moving forward.
> 2. In the past we have been more generous of spawning projects in
> Kubeflow umbrella, and we should now put higher bar and guidelines
> with a year roadmap before something becomes part of Kubeflow. And
> also be ok with "retiring" projects if they are not being invested in.

I agree completely on doing fewer things well. And also agree on
retiring projects that are unmaintained. This is why I mentioned that we
are not going to be including everything in our proposal, some things
will be left out, and this is expected. We will focus on the ones we
know we can maintain and provide quality work, as we've proven to do the
past 3 years. I agree on the higher bar as well for new projects, but
I'm only referring to existing components at this point, nothing new.

> 3. Now coming to the "end-to-end" story. As is evident from the
> thread, there are two kinds of consumers - folks who are more
> interested in one or couple of the verticals (pipelines, serving,
> training etc. ) or folks who look at Kubeflow as a single horizontal
> entity, and distribute as such. There is room for both. While we have
> been succeeding with WGs on the "vertical" front, the "horizontal"
> story is sort of falling apart. We don't see representatives from
> respective organizations on community calls who would be interested in
> carrying that mantle forward.
> 4. The best way to tackle this can be to find consumers and users who
> benefit by positioning Kubeflow as a distro, and by virtue of it are
> the ones they carry the mantle forward on that story. On the other
> hand, folks who are more interested in verticals and
> consuming/redistributing them, should carry the burden there.

I agree that there is room for both and this is a good thing. This is
why we stepped up at this point to make sure the horizontal, end-to-end
story doesn't fall apart, judging from the progress the last couple of
months. At Arrikto, we are heavily invested in this, we have large
enterprise customers using the end-to-end platform, so this comes after
a lot of thought.

> 5. In lieu of not getting "owners" who are invested in the horizontal
> story, its best to start communicating and setting the right
> expectations to the end user community. by positioning some parts of
> Kubeflow as "reference architectures", with "sample" implementations,
> instead of "solid supported implementations"
>

I think that we will not have to fall back to this route now, and we
will be able to keep the end user community vibrant and satisfied going
forward. So, we will be coming back with a written proposal, and we'd
love to get your feedback once everything becomes specific.

Thank you,
Constantinos


> On #5 above, if we do focus the story on vertical more, as opposed to
> horizontal, agree with Jeremy's proposal to set some minimum criteria
> for Kubeflow applications.
>
> 1. Every component/application to provide and support a KFP component
> which can be used in Pipelines. This will be a low-touch cohesive way
> for applications to opt-in the "Pipeline Train". Creating a component
> for KFP is well scoped and nicely defined.
> 2. If we can define clarity around how to plug in "Metadata", for each
> component in a standard way, this can also be made a minimum.
> 3. in addition to each WG providing a Kustomize configuration for
> deployment of the app from the WG 
>
> Beyond that, if we do find a solid work groups owning a central
> dashboard, auth/authz, ingress etc, end to end installer etc, rest of
> the criteria can be laid out in an "optional" fashion.
>
> Thanks,
>
> Animesh
>
>
> On Wed, Oct 7, 2020 at 1:45 PM Michael Tanenbaum <mtane...@d2iq.com
> <mailto:mtane...@d2iq.com>> wrote:
>
> Hi all -
>
> I would like to second the sentiments expressed by Constantinos.
>
> As a vendor currently selling an enterprise distro, we stand in
> support of Kubeflow /because/ it shows the greatest potential to
> >     <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Buns...@googlegroups.com>>
> >     >   
>  <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >     <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Buns...@googlegroups.com>>>.
> >     >     To view this discussion on the web visit
> >     >   
> >   
>   https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com>
> >   
>  <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com>>
> >     >   
> >   
>   <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >   
>  <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >     >
> >     > --
> >     > You received this message because you are subscribed
> to the Google
> >     > Groups "kubeflow-discuss" group.
> >     > To unsubscribe from this group and stop receiving
> emails from
> >     it, send
> >     > an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >     <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Buns...@googlegroups.com>>
> >     > <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >     <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Buns...@googlegroups.com>>>.
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/63dde059-37b6-507d-aeef-6cd8aa13426d%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/63dde059-37b6-507d-aeef-6cd8aa13426d%40arrikto.com>.
>
>
>
> --
> *Michael Tanenbaum*
> Principal Sales Engineer, D2iQ
> +1-917-512-3660
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

Mathew Wicks

unread,
Oct 12, 2020, 8:21:39 PM10/12/20
to kubeflow-discuss
Hi All,

I strongly believe if Kubeflow wants to be viewed as the industry standard Kubernetes Data platform we must have something beyond the name which ties the components together.

I view this 'glue' as being three main things:
   1. A common user-story
   2. A common distribution
   3. A common UI (+ single sign-on)

------------------------------
1 - A common user-story
------------------------------
Clarification:
  • There should be a set of clearwell-defined, and opinionated user-stories for Kubeflow
  • Each core application should be able to justify its contribution to at least one of the user-stories

------------------------------
2 - A common distribution
------------------------------
Clarification:
  • The Kubeflow Project should support a single official distribution (for existing Kuberntes clusters only)
  • To preserve the idea of Kubeflow as a platform, components of Kubeflow should not brand themselves as Kubeflow (even if they can be installed independently)
Current issues:
  • We are effectively unable to release new versions (especially patch releases) of Kubeflow, because there are too many dependant distributions which are NOT controlled by the project itself
  • We are distracted by our attempts to provide Kubernetes Clusters + Kubeflow, this has led to us neglecting Kubeflow development, and fragmenting into multiple forks/distributions
  • Users are confused when they see things like Kubeflow Pipelines, and assume that is the entire Kubeflow platform
Pending Proposals:

------------------------------
3 - A common UI
------------------------------
Clarification:
  • Each major user-facing component of Kubeflow should have at least some form of UI element
  • Every UI exposed by Kubeflow should be only accessible through the secure central dashboard
Current issues:
  • Normal users are unaware of all the components/features of Kubeflow which are not present in the UI. (For example, Profiles)
  • Some UI components are not scoped to the currently logged-in user (meaning users can see ALL namespaces)

William Teo

unread,
Oct 13, 2020, 3:59:10 AM10/13/20
to Mathew Wicks, kubeflow-discuss
Hi Mathew

I have a bit of issue with the following statement. 
  • Every UI exposed by Kubeflow should be only accessible through the secure central dashboard
Mostly because as mentioned previously, there are 2 types of users. Those that use only some components of kubeflow, and those who view kubeflow as a whole.

For e.g., I would want to be able to use kfp independently from kubeflow per se - esp when I am running a very light weight distribution for a small team.

The part I like about kubeflow is that it is easy for me to integrate third party (or in-house) components into the kubeflow "framework".

Although I would hope central dashboard can be modified to be more composible so that I can customize w/o maintaining a fork.


To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com.

Mathew Wicks

unread,
Oct 13, 2020, 4:12:36 AM10/13/20
to kubeflow-discuss
Hey William,

If my proposal were to be accepted by Kubeflow, nothing is to prevent Pipelines from being made available outside of the full Kubeflow platform, (with the caveat that Kubeflow Pipelines would need to rebrand itself to not use the word Kubeflow, as Kubeflow is the entire platform).

However, a better solution for your use-case would be to install the generic distribution and only select the Pipelines component. 

Naturally, the generic distribution would be modular, with only some components required, like for example, the Central Dashboard (assuming you wanted any form of WebUI).

Jeremy Lewi

unread,
Oct 15, 2020, 9:34:22 AM10/15/20
to Mathew Wicks, kubeflow-discuss
I expanded on this a bit more in:  http://bit.ly/kna-proposal

I agree that we need a cohesive platform. I think that cohesion should come from metadata.

For example, no matter how a user trains a model (in a notebook, via kubectl, as part of a pipeline) metadata about that should be logged and viewable consistently.

To demo the cohesiveness of KF what if users went to the metadata UI and saw data gathered from all their applications (Katib, Pipelines, TFJob, notebooks/kale)?

Deployment and application management is a big problem; is it a kubeflow problem? Or is it an industry/kubernetes problem?

Kubernetes has apps-sig which is focused on running and deploying applications.  Does Kubeflow need its own WG for deployment/app management? Rather than creating our own WG should we direct folks to participate in apps-sig?

J


Luke Marsden

unread,
Oct 15, 2020, 3:44:13 PM10/15/20
to Jeremy Lewi, Mathew Wicks, kubeflow-discuss
Metadata consistency is great, but I personally think that ease of deployment is a much more fundamental need for the success of the project.

If users can't easily try out the platform and deploy the platform in production, all the metadata consistency in the world won't help the fact that no one can use it. :)

Developing kubeadm helped adoption of Kubernetes, which was way too hard to install before that. Let's not have kubeflow go backwards on that point. We can't rely on vendors to distribute kubeflow.

Just my 2c.

Cheers,
Luke



--
Luke Marsden   |   Founder & CEO, MLOps Consulting

Constantinos Venetsanopoulos

unread,
Oct 21, 2020, 8:48:06 PM10/21/20
to kubeflow-discuss
Hello all,

As we discussed in this email thread, and in the last community meeting,
please find below our proposal for ensuring Kubeflow can continue being
released as a coherent platform with simple, portable, and scalable
end-to-end workflows:

https://docs.google.com/document/d/1BpR4RTp4m2GO_r-c3_xZ-qGnCfvQdqiRh4V0cFUSBqA/

It is evident that this entails a significant amount of effort, both in
development work and project management. We will commit to do it,
assuming that we have a strong backing from the community, which also
realizes this vision.

Once we have consensus, to prove that we are up to it, we will commit to
drive the next release of Kubeflow, after 1.2 gets released, based on
the deliverables described in the proposal.

We are looking forward to your comments and hopefully we will all be
successful in making Kubeflow the de facto ML platform on Kubernetes.

Thank you,
Constantinos

Jeremy Lewi

unread,
Oct 21, 2020, 11:33:02 PM10/21/20
to Constantinos Venetsanopoulos, kubeflow-discuss
Thanks Constantinos; Can you enable commenting for kubeflow-discuss?

J

To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubeflow-discuss/d02ce379-c5e6-4809-13cd-0eaf168b1423%40arrikto.com.

jo...@pattersonconsultingtn.com

unread,
Oct 22, 2020, 9:31:19 AM10/22/20
to kubeflow-discuss
I'd like to echo what Luke is saying here; for adoption purposes, the "hello world"-aspect of Kubeflow needs to be as simple as possible.

right now that's not the case.

Towards that end: If pipelines and metadata are to be foundational to Kubeflow going forward, then I'd propose a command-line script included with kubeflow that let's a data scientist quickly mock up a pipeline and run it locally (without installing KF or K8s). 

That way, pipelines can gain adoption quickly for the python / pandas crowd as an easy workflow engine --- that has the upside of the full Kubeflow platform behind it the moment they need the full deal.

JP

Patterson Consulting

David Aronchick

unread,
Oct 22, 2020, 11:48:16 AM10/22/20
to jo...@pattersonconsultingtn.com, kubeflow-discuss
Hey Josh--

Would that mean doing it all in a language (eg Python)? Or could K3s be acceptable?

Jeremy Lewi

unread,
Oct 22, 2020, 12:23:03 PM10/22/20
to David Aronchick, jo...@pattersonconsultingtn.com, kubeflow-discuss
I think everyone agrees that ease of deployment and a great day 0 as well as day 2 experience is important. 

The question IMO:  Is the focus on deployment distracting us as community from AI/ML? 

I think it is. I would much rather we as a community spend our time debating questions like

* How should applications emit metadata consistently
* What should the schemas be for models / datasets/ pipelines etc...
* How do you design custom resources so they are easily orchestrable with pipelines.
* How do you monitor model performance

Then debating questions like
  • What middle ware should be included (ISTIO? Dex? minio?....)?
  • What declarative application management tool chain should be used (kfctl, kpt, kudo, juju, etc...)
There are already numerous groups focused on solving the latter. I don't think we are uniquely positioned to address the latter but I do think we do have a lot to add to the former.

A good day 0 experience will all be different on a GCP, AWS, MiniKube, a generic multi-node K8s conformant cluster, etc...

When it comes to deployment, we have followed an approach of embracing this diversity and encouraging a thousand flowers to bloom.

I think we should formalize this by saying 

1) A Kubeflow distribution is an opinionated bundle of applications optimized for a particular environment or use case
2) All Kubeflow distributions should be developed outside of Kubeflow and treated equally.

So if Arrikto wants to develop another opinionated distribution (it already has MiniKF) I think that's great. If others want to collaborate on that; even better. But I think that should happen outside Kubeflow.

The closest thing we have to an industry standard is publishing piles of YAML containing Kubernetes resources that can be piped to kubectl. I think as a community we should focus on publishing 
YAML for each application consistently and leave the rest for the distribution owners.

J





Josh Patterson

unread,
Oct 22, 2020, 1:00:43 PM10/22/20
to David Aronchick, kubeflow-discuss
David,
I think all of that is up in the air -- I'm just looking at it (right now, shooting from the hip) in terms of the "hello world" UX of it all. 

e.g., "How do we do it with the minimal number of steps for a 1st time user?"

fwiw, k3s seems pretty easy, but I've yet to actually run it.

But If I'm writing a wish list here, we'd want to do:

(1) install: curl / yum / pip / blah install (30s)
(2) hack: edit up a my_pipeline.py (2m)
(3) run: kf_pipelines my_pipeline.py (30s)

and just keep it simple so that the new user isnt thinking that hard about the underlying stuff, about containers, etc -- but once they get farther along, they go "oh, this just runs on k8s? and GCP? and Azure? cool, time to call DevOps to maybe standup more stuff or just log into Azure..."

under the hood, I think a "kf_pipelines" runner script could either "do it all in python land" or "do the k3s" setup for us (prompt: you dont want k3s, can we install it quickly? [y/n]" --- as long as it "just worked", i think thats more of a tactical / execution decision.

JP
--

Josh Patterson

unread,
Oct 22, 2020, 1:03:11 PM10/22/20
to David Aronchick, kubeflow-discuss
Extending this line of thought further (now that I think about it), the concept of the "quick install kf_pipelines" is analogous to k3s, really:

as "K3s is to K8s", "kf_pipeline would be to kubeflow" in this comparison

(maybe even call it "k3f") =D
--

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 1:07:56 PM10/22/20
to Jeremy Lewi, kubeflow-discuss
Sure! Done.

Constantinos
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>     <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
>  <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "kubeflow-discuss" group.
> >> To unsubscribe from this group and stop receiving emails from
> it, send
> >> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxbFybORrBQXP8iAYRW6K8aur9aiZJdkiabVj2hZaMtw%3Dg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/d02ce379-c5e6-4809-13cd-0eaf168b1423%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/d02ce379-c5e6-4809-13cd-0eaf168b1423%40arrikto.com>.
>

David Aronchick

unread,
Oct 22, 2020, 1:30:59 PM10/22/20
to Josh Patterson, Jeremy Lewi, kubeflow-discuss
@jo...@pattersonconsultingtn.com I think that not having SOME local experience based on Kubernetes will cause eventual drift. I think k3s or miniKF or whatever is critical to ensure compatibility.

@Jeremy Lewi Agreed that we're SOMEWHAT being distracted from AI/ML. HOWEVER, because setup & maintenance is such a huge problem, it's also preventing a lot of innovation later because it's (effectively) gatekeeping from core. I don't think whatever the default is should be based on a single company's platform, but it should be good (for some definition of good) out of the box with a core set of features that are well supported and tested. I think we all agree that that is not the case today.

Your other questions can be addressed in parallel! (Ideally, as a joint effort - e.g. default components should emit/consume structured metadata, we have templates that let people write custom components that use these tools, etc)

But I can say, from the experience with Kubernetes, that we had a big pivot to broader use with the work behind kubeadm for simple setup. We should just make sure that we have a similarly base experience here - I wouldn't call it a "distribution" but it should be some form of commitment to a great out of the box experience.

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 1:43:59 PM10/22/20
to Jeremy Lewi, David Aronchick, jo...@pattersonconsultingtn.com, kubeflow-discuss
Hello Jeremy,

In general I agree with most of the comments on the AI/ML focus.

Regarding distros:

We are not discussing about an opinionated distribution here. MiniKF is
indeed an opinionated distribution, since it contains customizations to
Kubeflow that haven't yet been upstreamed, and Arrikto's Rok, which is
proprietary. The goal for this effort is to just provide a way to deploy
the current open source components of Kubeflow and stitch them together
as happened in the past. Essentially replicate mostly Google's previous
effort, which everyone in the community loved. This is meant to be a
base to help people create distros, as Animesh commented in the previous
community meeting. This way, they won't have to re-invent, and re-test
everything every time. So, if the community agrees that this is useful,
I think it should definitely be part of Kubeflow.

Best,
Constantinos

On 22/10/20 09:22, Jeremy Lewi wrote:
> I think everyone agrees that ease of deployment and a great day 0 as
> well as day 2 experience is important. 
>
> The question IMO:  Is the focus on deployment distracting us as
> community from AI/ML? 
>
> I think it is. I would much rather we as a community spend our time
> debating questions like
>
> * How should applications emit metadata consistently
> * What should the schemas be for models / datasets/ pipelines etc...
> * How do you design custom resources so they are easily orchestrable
> with pipelines.
> * How do you monitor model performance
>
> Then debating questions like
>
> * What middle ware should be included (ISTIO? Dex? minio?....)?
> * What declarative application management tool chain should be used
> <mailto:jo...@pattersonconsultingtn.com>
> <jo...@pattersonconsultingtn.com
> <mailto:jo...@pattersonconsultingtn.com>> wrote:
>
> I'd like to echo what Luke is saying here; for adoption
> purposes, the "hello world"-aspect of Kubeflow needs to be as
> simple as possible.
>
> right now that's not the case.
>
> Towards that end: If pipelines and metadata are to be
> foundational to Kubeflow going forward, then I'd propose a
> command-line script included with kubeflow that let's a data
> scientist quickly mock up a pipeline and run it locally
> (without installing KF or K8s). 
>
> That way, pipelines can gain adoption quickly for the python /
> pandas crowd as an easy workflow engine --- that has the
> upside of the full Kubeflow platform behind it the moment they
> need the full deal.
>
> JP
>
> Patterson Consulting
> www.pattersonconsultingtn.com
> <http://www.pattersonconsultingtn.com>
>
> On Thursday, October 15, 2020 at 3:44:13 PM UTC-4 Luke Marsden
> wrote:
>
> Metadata consistency is great, but I personally think that
> ease of deployment is a /much/ more fundamental need for
> the success of the project.
>
> If users can't easily try out the platform and deploy the
> platform in production, all the metadata consistency in
> the world won't help the fact that no one can use it. :)
>
> Developing kubeadm
> <https://kubernetes.io/blog/2016/09/how-we-made-kubernetes-easy-to-install/>
> helped adoption of Kubernetes, which was way too hard to
> install before that. Let's not have kubeflow go backwards
> on that point. We can't rely on vendors to distribute
> kubeflow.
>
> Just my 2c.
>
> Cheers,
> Luke
>
> On Thu, 15 Oct 2020 at 14:34, Jeremy Lewi <jer...@lewi.us>
> wrote:
>
> I expanded on this a bit more
> in:  http://bit.ly/kna-proposal
> <http://bit.ly/kna-proposal>
>
> I agree that we need a cohesive platform. I think that
> cohesion should come from metadata.
>
> For example, no matter how a user trains a model (in a
> notebook, via kubectl, as part of a pipeline) metadata
> about that should be logged and viewable consistently.
>
> To demo the cohesiveness of KF what if users went to
> the metadata UI and saw data gathered from all their
> applications (Katib, Pipelines, TFJob, notebooks/kale)?
>
> Deployment and application management is a big
> problem; is it a kubeflow problem? Or is it an
> industry/kubernetes problem?
>
> Kubernetes has apps-sig
> <https://github.com/kubernetes/community/blob/master/sig-apps/README.md> which
> is focused on running and deploying applications. 
> Does Kubeflow need its own WG for deployment/app
> management? Rather than creating our own WG should we
> direct folks to participate in apps-sig?
>
> J
>
>
> On Tue, Oct 13, 2020 at 1:12 AM Mathew Wicks
> <mathew...@gmail.com> wrote:
>
> Hey William,
>
> If my proposal were to be accepted by Kubeflow,
> nothing is to prevent Pipelines from being made
> available outside of the full Kubeflow platform,
> (with the caveat that Kubeflow Pipelines would
> need to rebrand itself to not use the word
> Kubeflow, as Kubeflow is the entire platform).
>
> However, a better solution for your use-case would
> be to install the *generic distribution *and only
> select the Pipelines component. 
>
> Naturally, the generic distribution would be
> modular, with only some components required, like
> for example, the Central Dashboard (assuming you
> wanted any form of WebUI).
>
> On Tuesday, 13 October 2020 at 6:59:10 pm UTC+11
> etern...@gmail.com wrote:
>
> Hi Mathew
>
> I have a bit of issue with the following
> statement. 
>
> * Every UI exposed by Kubeflow should
> be* only accessible* through the secure
> central dashboard
>
> Mostly because as mentioned previously, there
> are 2 types of users. Those that use only some
> components of kubeflow, and those who view
> kubeflow as a whole.
>
> For e.g., I would want to be able to use kfp
> independently from kubeflow per se - esp when
> I am running a very light weight distribution
> for a small team.
>
> The part I like about kubeflow is that it is
> easy for me to integrate third party (or
> in-house) components into the kubeflow
> "framework".
>
> Although I would hope central dashboard can be
> modified to be more composible so that I can
> customize w/o maintaining a fork.
>
>
> On Tue, 13 Oct 2020, 8:21 am Mathew Wicks,
> <mathew...@gmail.com> wrote:
>
> Hi All,
>
> I strongly believe if Kubeflow wants to be
> viewed as the industry standard Kubernetes
> Data platform we must have
> something *beyond* *the name *which ties
> the components together.
> *
> *
> *I view this 'glue' as being three main
> things:*
> *   *1.* *A common user-story
>    2. A common distribution
>    3. A common UI (+ single sign-on)
>
> *------------------------------*
> *1 - A common user-story*
> *------------------------------*
> *Clarification:*
>
> * There should be a set
> of *_clear_*, *_well-defined_*,
> and *_opinionated_* user-stories for
> Kubeflow
> * Each *core* *application* should be
> able to justify its contribution to at
> least one of the user-stories
>
> *
> *
> *------------------------------*
> *2 - A common distribution*
> *------------------------------*
> *Clarification:*
>
> * The Kubeflow Project should support
> a *single official distribution *(for
> existing Kuberntes clusters only)
> * To preserve the idea of *Kubeflow as a
> platform*, components of Kubeflow
> should *not* brand themselves as
> Kubeflow (even if they can be
> installed independently)
>
> *Current issues:*
>
> * We are effectively unable to release
> new versions (especially patch
> releases) of Kubeflow, because there
> are too many dependant distributions
> which are NOT controlled by the
> project itself
> * We are distracted by our attempts to
> provide Kubernetes Clusters +
> Kubeflow, this has led to us
> neglecting Kubeflow development, and
> fragmenting into multiple
> forks/distributions
> * Users are confused when they see
> things like Kubeflow Pipelines, and
> assume that is the entire Kubeflow
> platform
>
> *Pending Proposals:*
>
> * Generic Distribution:
> https://github.com/kubeflow/manifests/issues/1554
> <https://github.com/kubeflow/manifests/issues/1554>
>
> *
> *
> *------------------------------*
> *3 - A common **UI*
> *------------------------------*
> *Clarification:*
>
> * Each major user-facing component of
> Kubeflow should have *at least some
> form *of UI element
> * Every UI exposed by Kubeflow should
> be*only accessible* through the secure
> central dashboard
>
> *Current issues:*
>
> * Normal users are unaware of all the
> components/features of Kubeflow which
> are not present in the UI. (For
> example, Profiles)
> * Some UI components are not scoped to
> > +1-917-512-3660 <tel:(917)%20512-3660>
> > --
> > You received this message because
> you are subscribed to the Google
> > Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and
> stop receiving emails from it,
> > send an email to
> kubeflow-discu...@googlegroups.com
> >
> <mailto:kubeflow-discu...@googlegroups.com>.
>
> > To view this discussion on the web
> visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> >
>
> --
> You received this message because you are
> subscribed to the Google Groups
> "kubeflow-discuss" group.
>
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> kubeflow-discu...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are
> subscribed to the Google Groups "kubeflow-discuss"
> group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
> kubeflow-discu...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed
> to the Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
> kubeflow-discu...@googlegroups.com.
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Luke Marsden   |   Founder & CEO, MLOps Consulting
> US  +1 415-231-8523 <tel:(415)%20231-8523>
> UK  +44 7791750420 <tel:+44%207791%20750420>
>
> --
> You received this message because you are subscribed to the
> Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Darryl Ruggles

unread,
Oct 22, 2020, 2:13:36 PM10/22/20
to kubeflow-discuss
Exactly. We really need a clear way to install a non-proprietary distribution of the main Kubeflow components. Without this people get very frustrated trying to figure out how to setup Kubeflow and use it.

Jeremy Lewi

unread,
Oct 22, 2020, 3:29:27 PM10/22/20
to Darryl Ruggles, kubeflow-discuss
Hi Constantinos

With respect to your specific proposal do you have any co-sponsors for the proposals? Who else would you be working with to maintain it?

J

Rui Vasconcelos

unread,
Oct 22, 2020, 3:32:05 PM10/22/20
to kubeflow-discuss
Hi everyone!

It seems that a lot of people in the community have a viable e2e platform, day-0 and day-2 stories - Arrikto, AgileStacks, D2iQ, Canonical, OpenShift Operator - but these are not visible to the end-user seeing KF for the first time, and this is hurting the community overall.

Raising a few questions :)
  • Given that these efforts sit outside of Kubeflow, should we create a distro list in the docs to help users find e2e deployment methods?
  • Alternatively, should there be a commonplace to host distributions?
  • If neither of the above, does kfctl solve the end-user problem of simply trying Kubeflow? Can we make it so?
I would avoid arguing what is the best 3rd party distro or try to make one the default upstream way, I believe we all benefit from open collaboration.

Rui



--
Rui Vasconcelos
Product Manager - AI/ML
Canonical | Ubuntu
+351 919154539

Jeremy Lewi

unread,
Oct 22, 2020, 4:05:08 PM10/22/20
to Rui Vasconcelos, kubeflow-discuss
> I would avoid arguing what is the best 3rd party distro or try to make one the default upstream way, I believe we all benefit from open collaboration.

+1

We have always endeavored to be neutral and not endorse one distribution over another. I think we should continue to push in this direction. Users should be presented with a list of options and choose the one that works best for them.

J

David Aronchick

unread,
Oct 22, 2020, 5:11:57 PM10/22/20
to Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
+1 as well. Anything we did as a default should be upstreamed and completely neutral. If folks want to do distributions (which should be encouraged!), there should be a core way to extend this base - but not in the core distribution.

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 6:04:06 PM10/22/20
to Jeremy Lewi, kubeflow-discuss
Hello Jeremy,

Our plan was to first get consensus from the community that this is
something that is actually desirable, and get comments/feedback on the
proposal before trying to find a co-lead.

Since, no one had stepped up out until now, we were prepared to do this
by ourselves. Note that we will need at least four leads from our side
to be able to handle something like this, and make sure there is
redundancy and multiple people accountable.

From the Arrikto side we will need to have: Yannis Zarkadas, Stefano
Fioravanzo, Elias Katsakioris, and Vangelis Koukis.

We would also love to work with the AWS team, and collaborate on a CI
process to test on the new infrastructure. If not, we will have to do
this as well.

Thus, we would like to get a pulse from the community first, we already
have some positive answers it seems, and if someone else steps up to put
actual development work on this, we would be happy to consider them as a
co-lead. But at this point, they should definitely have a proven track
record of successful upstream contributions to the parts of the code
that this proposal touches, to be considered as a co-lead.

We will also be coming back to the comments on the doc, we just want to
give some time for people to go through it.

Hope this is helpful.

Best,
Constantinos
> > <http://www.pattersonconsultingtn.com
> > > +1-917-512-3660 <tel:(917)%20512-3660> <tel:(917)%20512-3660>
> <https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are
> > subscribed to the Google Groups "kubeflow-discuss"
> > group.
> > To unsubscribe from this group and stop receiving
> > emails from it, send an email to
> > kubeflow-discu...@googlegroups.com.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are subscribed
> > to the Google Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving
> > emails from it, send an email to
> > kubeflow-discu...@googlegroups.com.
> >
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> >
> >
> >
> > --
> > Luke Marsden   |   Founder & CEO, MLOps Consulting
> > US  +1 415-231-8523 <tel:(415)%20231-8523>
> <tel:(415)%20231-8523>
> > UK  +44 7791750420 <tel:+44%207791%20750420>
> <tel:+44%207791%20750420>
> >
> > --
> > You received this message because you are subscribed to the
> > Google Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving emails from
> > it, send an email to
> > kubeflow-discu...@googlegroups.com
> > <mailto:kubeflow-discu...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving emails
> from it,
> > send an email to kubeflow-discu...@googlegroups.com
> > <mailto:kubeflow-discu...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaW4-t00Kp8r9YeAq7rZjvGZTwWF191yvEb0%3D45DmGvqA%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaW4-t00Kp8r9YeAq7rZjvGZTwWF191yvEb0%3D45DmGvqA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 6:09:16 PM10/22/20
to David Aronchick, Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
+1

Note that this proposal is not meant to act as a vendor distribution. It
strives for a completely open source way to deploy the official
components together. It is supposed to act as a basis to help people
install, and vendors package/create distributions.

Note that Arrikto already provides a scale-out, enterprise Kubeflow
distribution, but this is something completely different, and not really
related to this proposal.

Thank you,
Constantinos

On 22/10/20 14:11, David Aronchick wrote:
> +1 as well. Anything we did as a default should be upstreamed and
> completely neutral. If folks want to do distributions (which should be
> encouraged!), there should be a core way to extend this base - but not
> in the core distribution.
>
> On Thu, Oct 22, 2020 at 1:05 PM Jeremy Lewi <jer...@lewi.us
> <mailto:jer...@lewi.us>> wrote:
>
> > I would avoid arguing what is the best 3rd party distro or try
> to make one the default upstream way, I believe we all benefit
> from open collaboration.
>
> +1
>
> We have always endeavored to be neutral and not endorse one
> distribution over another. I think we should continue to push in
> this direction. Users should be presented with a list of options
> and choose the one that works best for them.
>
> J
>
> On Thu, Oct 22, 2020 at 12:32 PM Rui Vasconcelos
> <rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>> wrote:
>
> Hi everyone!
>
> It seems that a lot of people in the community have a viable
> e2e platform, day-0 and day-2 stories - Arrikto, AgileStacks,
> D2iQ, Canonical, OpenShift Operator - but these are not
> visible to the end-user seeing KF for the first time, and this
> is hurting the community overall.
>
> Raising a few questions :)
>
> * Given that these efforts sit outside of Kubeflow, should
> we create a distro list in the docs to help users find e2e
> deployment methods?
> * Alternatively, should there be a commonplace to host
> distributions?
> * If neither of the above, does kfctl solve the end-user
> > <http://www.pattersonconsultingtn.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are
> > subscribed to the Google Groups "kubeflow-discuss"
> > group.
> > To unsubscribe from this group and stop receiving
> > emails from it, send an email to
> > kubeflow-discu...@googlegroups.com.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are subscribed
> > to the Google Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving
> > emails from it, send an email to
> > kubeflow-discu...@googlegroups.com.
> >
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> >
> >
> >
> > --
> > Luke Marsden   |   Founder & CEO, MLOps Consulting
> > US  +1 415-231-8523 <tel:(415)%20231-8523>
> <tel:(415)%20231-8523>
> > UK  +44 7791750420 <tel:+44%207791%20750420>
> <tel:+44%207791%20750420>
> >
> > --
> > You received this message because you are subscribed
> to the
> > Google Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving
> emails from
> > it, send an email to
> > kubeflow-discu...@googlegroups.com
> > <mailto:kubeflow-discu...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> >
> > --
> > You received this message because you are subscribed
> to the Google
> > Groups "kubeflow-discuss" group.
> > To unsubscribe from this group and stop receiving
> emails from it,
> > send an email to kubeflow-discu...@googlegroups.com
> > <mailto:kubeflow-discu...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to
> the Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Rui Vasconcelos
> Product Manager - AI/ML
> *Canonical | **Ubuntu*
> +351 919154539
> --
> You received this message because you are subscribed to the
> Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Work

unread,
Oct 22, 2020, 6:16:58 PM10/22/20
to David Aronchick, Jeremy Lewi, Constantinos Venetsanopoulos, Rui Vasconcelos, kubeflow-discuss
Hi Constantinos, 

This is Yao, from AWS. From AWS side, we’re happy to work with you for setting up CI process to test on shared test-infra. There are already some POC work ongoing. 

Thanks,
Yao

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 6:31:11 PM10/22/20
to Work, David Aronchick, Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
Hello Yao,

This is great! We'd definitely like that, thank you for your support.

Best,
Constantinos

Animesh Singh

unread,
Oct 22, 2020, 8:12:28 PM10/22/20
to Constantinos Venetsanopoulos, Work, David Aronchick, Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
Thanks all.

This thread went in many directions, so summarizing the thread based on my understanding so far

1. Each application WG producing a Pipeline Component and Metadata to be plugged into KFP with every release
+1 on above

2. A core distribution of Kubeflow distributing applications based on core WGs (Notebooks, Training, AutoML, Serving and Pipelines) on vanilla OSS Kubernetes, Istio and KNative 
 +1 on above.

3. AWS helping with testing infra for this (I assume this will use Kubernetes OSS, as opposed to EKS). 
+1 on above

4. Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto, Seldon...) are not part of this WG, and need to be managed independently. 
+1 on above

4. A new Manifests repo, no default deployment tool (e.g. kfctl) as part of new Control Plane WG
This is the only area where I need clarity. So KFCTL and existing Manifests repo are going to be managed by Deployment WG? If yes, should it be called something else (e.g. Distributions WG)?

Thanks,

Animesh


Jeremy Lewi

unread,
Oct 22, 2020, 9:07:06 PM10/22/20
to Animesh Singh, Constantinos Venetsanopoulos, Work, David Aronchick, Rui Vasconcelos, kubeflow-discuss
A core distribution of Kubeflow distributing applications based on core WGs (Notebooks, Training, AutoML, Serving and Pipelines) on vanilla OSS Kubernetes, Istio and KNative 
Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto, Seldon...) are not part of this WG, and need to be managed independently. 

The idea of a "core"/default/official distribution seems problematic. Is there a reason we can't treat all distributions uniformly?

All distributions are opinionated. Whether you are using only OSS components or not; every distribution represents a set of opinionated decisions.

Anything which creates a perception that one distribution is endorsed by Kubeflow over others seems inherently problematic to me.
A WG says Kubeflow is responsible for this distribution and that seems problematic to me. 

Why do we need to create WGs to sponsor distributions? This just seems like an invitation for conflict as various groups agitate to advance their agenda by getting their opinions upstreamed.
Adopting only OSS tech doesn't preempt this (emacs vs. vim, helm vs kustomize, etc...)

If folks want to collaborate on a distribution, why don't they just pick a name spin up a GitHub repo and invite interested parties to collaborate? If we adopt a policy of no distributions within Kubeflow then we carve out a clear space for an ecosystem of distributions.








Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 9:13:27 PM10/22/20
to Animesh Singh, Work, David Aronchick, Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
Hello Animesh,

Thank you for summarizing, I think you are 100% accurate.

To your 4th question:

Yes. KFCTL is an opinionated deployment tool, which means it should be
maintained (along with its manifests) by the vendors/platform owners
that actually use it for their opinionated distros. I don't really have
an opinion on whether the WG should be renamed or not, I would let the
vendors who use it decide how it should be maintained, and whether they
need a WG to maintain it.

Best,
Constantinos

On 22/10/20 17:12, Animesh Singh wrote:
> Thanks all.
>
> This thread went in many directions, so summarizing the thread based
> on my understanding so far
>
> 1. *Each application WG producing a Pipeline Component and Metadata to
> be plugged into KFP with every release*
> +1 on above
>
> 2. *A core distribution of Kubeflow distributing applications based on
> core WGs (Notebooks, Training, AutoML, Serving and Pipelines) on
> vanilla OSS Kubernetes, Istio and KNative *
>  +1 on above.
>
> 3.*AWS helping with testing infra for this (I assume this will use
> Kubernetes OSS, as opposed to EKS). *
> +1 on above
>
> 4. *Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto,
> Seldon...) are not part of this WG, and need to be managed
> independently.* 
> +1 on above
>
> 4. *A new Manifests repo, no default deployment tool (e.g. kfctl) as
> part of new Control Plane WG*
> This is the only area where I need clarity. So KFCTL and existing
> Manifests repo are going to be managed by Deployment WG
> <https://github.com/kubeflow/community/pull/402>? If yes, should it be
> called something else (e.g. Distributions WG)?
>
> Thanks,
>
> Animesh
>
>
> On Thu, Oct 22, 2020 at 3:31 PM Constantinos Venetsanopoulos
> <cv...@arrikto.com <mailto:cv...@arrikto.com>> wrote:
>
> Hello Yao,
>
> This is great! We'd definitely like that, thank you for your support.
>
> Best,
> Constantinos
>
> On 22/10/20 15:16, Work wrote:
> > Hi Constantinos, 
> >
> > This is Yao, from AWS. From AWS side, we’re happy to work with
> you for
> > setting up CI process to test on shared test-infra. There are
> already
> > some POC work ongoing. 
> >
> > Thanks,
> > Yao
> > On Oct 22, 2020, 3:09 PM -0700, Constantinos Venetsanopoulos
> > <cv...@arrikto.com <mailto:cv...@arrikto.com>>, wrote:
> >> +1
> >>
> >> Note that this proposal is not meant to act as a vendor
> distribution. It
> >> strives for a completely open source way to deploy the official
> >> components together. It is supposed to act as a basis to help
> people
> >> install, and vendors package/create distributions.
> >>
> >> Note that Arrikto already provides a scale-out, enterprise Kubeflow
> >> distribution, but this is something completely different, and
> not really
> >> related to this proposal.
> >>
> >> Thank you,
> >> Constantinos
> >>
> >> On 22/10/20 14:11, David Aronchick wrote:
> >>> +1 as well. Anything we did as a default should be upstreamed and
> >>> completely neutral. If folks want to do distributions (which
> should be
> >>> encouraged!), there should be a core way to extend this base -
> but not
> >>> in the core distribution.
> >>>
> >>> On Thu, Oct 22, 2020 at 1:05 PM Jeremy Lewi <jer...@lewi.us
> <mailto:jer...@lewi.us>
> >>> <mailto:jer...@lewi.us <mailto:jer...@lewi.us>>> wrote:
> >>>
> >>>> I would avoid arguing what is the best 3rd party distro or try
> >>> to make one the default upstream way, I believe we all benefit
> >>> from open collaboration.
> >>>
> >>> +1
> >>>
> >>> We have always endeavored to be neutral and not endorse one
> >>> distribution over another. I think we should continue to push in
> >>> this direction. Users should be presented with a list of options
> >>> and choose the one that works best for them.
> >>>
> >>> J
> >>>
> >>> On Thu, Oct 22, 2020 at 12:32 PM Rui Vasconcelos
> >>> <rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>
> >>> <mailto:rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>>> wrote:
> >>>
> >>> Hi everyone!
> >>>
> >>> It seems that a lot of people in the community have a viable
> >>> e2e platform, day-0 and day-2 stories - Arrikto, AgileStacks,
> >>> D2iQ, Canonical, OpenShift Operator - but these are not
> >>> visible to the end-user seeing KF for the first time, and this
> >>> is hurting the community overall.
> >>>
> >>> Raising a few questions :)
> >>>
> >>> * Given that these efforts sit outside of Kubeflow, should
> >>> we create a distro list in the docs to help users find e2e
> >>> deployment methods?
> >>> * Alternatively, should there be a commonplace to host
> >>> distributions?
> >>> * If neither of the above, does kfctl solve the end-user
> >>> problem of simply trying Kubeflow? Can we make it so?
> >>>
> >>> I would avoid arguing what is the best 3rd party distro or try
> >>> to make one the default upstream way, I believe we all benefit
> >>> from open collaboration.
> >>>
> >>> Rui
> >>>
> >>> On Thu, Oct 22, 2020 at 7:13 PM Darryl Ruggles
> >>> <rdar...@gmail.com <mailto:rdar...@gmail.com>
> >>>> <mailto:aron...@gmail.com <mailto:aron...@gmail.com>>> wrote:
> >>>>
> >>>> Hey Josh--
> >>>>
> >>>> Would that mean doing it all in a language (eg
> >>> Python)? Or could
> >>>> K3s be acceptable?
> >>>>
> >>>> On Thu, Oct 22, 2020 at 6:31 AM
> >>> jo...@pattersonconsultingtn.com
> <mailto:jo...@pattersonconsultingtn.com>
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>> <jer...@lewi.us <mailto:jer...@lewi.us>>
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>
> >>>>
> >>>>>>    
> >>>>  <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%252Buns...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>
> >>>>
> >>>>>>      >   
> >>>>>
> >>>>  <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>
> >>>>
> >>>>>>    
> >>>>  <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>
> >>>>
> >>>>>
> >>>>
> >>> <mailto:kubeflow-discuss%252Buns...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>>.
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>
> >>>>
> >>>>>>    
> >>>>  <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%252Buns...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>
> >>>>
> >>>>>>      >
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>
> >>>>
> >>>>>>    
> >>>>  <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>
> >>>>
> >>>>>
> >>>>
> >>> <mailto:kubeflow-discuss%252Buns...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>>.
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>.
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> >>>>
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Luke Marsden   |   Founder & CEO, MLOps Consulting
> >>>> US  +1 415-231-8523 <tel:(415)%20231-8523>
> >>> <tel:(415)%20231-8523>
> >>>> UK  +44 7791750420 <tel:+44%207791%20750420>
> >>> <tel:+44%207791%20750420>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the
> >>>> Google Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from
> >>>> it, send an email to
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the Google
> >>>> Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from it,
> >>>> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the Google
> >>>> Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from it, send
> >>>> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>
> >>> --
> >>> You received this message because you are subscribed to
> >>> the Google Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails
> >>> from it, send an email to
> >>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>>
> >>>
> >>> --
> >>> Rui Vasconcelos
> >>> Product Manager - AI/ML
> >>> *Canonical | **Ubuntu*
> >>> +351 919154539
> >>> --
> >>> You received this message because you are subscribed to the
> >>> Google Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails from
> >>> it, send an email to
> >>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails from it,
> >>> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails from
> it, send
> >>> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "kubeflow-discuss" group.
> >> To unsubscribe from this group and stop receiving emails from it,
> >> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> >> To view this discussion on the web visit
> >>
> https://groups.google.com/d/msgid/kubeflow-discuss/68e41875-66bf-fb35-a2de-d66a2d2482c7%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/68e41875-66bf-fb35-a2de-d66a2d2482c7%40arrikto.com>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/bfc408aa-1ab2-6273-1989-ae0b34b2de85%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/bfc408aa-1ab2-6273-1989-ae0b34b2de85%40arrikto.com>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CABMfuRvihMHbYCvYVN9AULUN2TsJKdeGj3KTuW6Zy06HotNMPg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CABMfuRvihMHbYCvYVN9AULUN2TsJKdeGj3KTuW6Zy06HotNMPg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Yao Xiao

unread,
Oct 22, 2020, 9:32:49 PM10/22/20
to Constantinos Venetsanopoulos, Animesh Singh, David Aronchick, Jeremy Lewi, Rui Vasconcelos, kubeflow-discuss
Hi Animesh, to clarify,

> 3. AWS helping with testing infra for this (I assume this will use Kubernetes OSS, as opposed to EKS).

For current Shared Test-infra work, it's relying on EKS. (It's not mandantory, WG who want to handle testing/release efforts by themselves, feel free to choose preferred solution including Github Actions / Travis / CircleCI / Self-managed Prow)

From EKS side, this is the same way that we did for CNCF previously.

If folks are looking for general Kubernetes OSS, it's another proposal compared to current Shared Test-infra proposal.

=========

Constantinos Venetsanopoulos

unread,
Oct 22, 2020, 9:35:18 PM10/22/20
to Jeremy Lewi, Animesh Singh, Work, David Aronchick, Rui Vasconcelos, kubeflow-discuss
It seems Jeremy and myself were typing at the same time... sorry about this

Please see comments inline:

On 22/10/20 18:06, Jeremy Lewi wrote:
> > *A core distribution of Kubeflow distributing applications based on
> core WGs (Notebooks, Training, AutoML, Serving and Pipelines) on
> vanilla OSS Kubernetes, Istio and KNative *
> *> **Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto,
> Seldon...) are not part of this WG, and need to be managed
> independently.* 
>
> The idea of a "core"/default/official distribution seems problematic.
> Is there a reason we can't treat all distributions uniformly?
>
> All distributions are opinionated. Whether you are using only OSS
> components or not; every distribution represents a set of opinionated
> decisions.
>

From the replies in this thread my understanding is that almost everyone
in the community agrees that this is not really an opinionated distro.
It is a completely neutral approach, same as we have been doing until KF
1.0. And it will actually help people create distros and advance the
ecosystem, treating it as a base.

> Anything which creates a perception that one distribution is endorsed
> by Kubeflow over others seems inherently problematic to me.
> A WG says Kubeflow is responsible for this distribution and that seems
> problematic to me. 
>
> Why do we need to create WGs to sponsor distributions? This just seems
> like an invitation for conflict as various groups agitate to advance
> their agenda by getting their opinions upstreamed.
> Adopting only OSS tech doesn't preempt this (emacs vs. vim, helm vs
> kustomize, etc...)
>

I don't think that this has to do with any kind of endorsement. And I
think agitation would be evident in this thread, if different groups
didn't like this approach. At this point we would be the first ones
backing out, if we saw that this is set up to fail due to blockers from
various groups of the community. I would give this a couple of days for
everyone to have time to go through it and express their opinion in the
community to make sure, but from the first reactions I'd say that
everyone here (including big vendors like AWS, IBM, Microsoft) seem very
positive, and agree that this should be part of Kubeflow and advances
the community.


> If folks want to collaborate on a distribution, why don't they just
> pick a name spin up a GitHub repo and invite interested parties to
> collaborate? If we adopt a policy of no distributions within Kubeflow
> then we carve out a clear space for an ecosystem of distributions.
>

I think that the ecosystem of distributions is already there, outside of
Kubeflow and this proposal doesn't really affect it. And if it does, it
is in a positive way.

Thank you,
Constantinos


> On Thu, Oct 22, 2020 at 5:12 PM Animesh Singh <animat...@gmail.com
> <mailto:animat...@gmail.com>> wrote:
>
> Thanks all.
>
> This thread went in many directions, so summarizing the thread
> based on my understanding so far
>
> 1. *Each application WG producing a Pipeline Component and
> Metadata to be plugged into KFP with every release*
> +1 on above
>
> 2. *A core distribution of Kubeflow distributing applications
> based on core WGs (Notebooks, Training, AutoML, Serving and
> Pipelines) on vanilla OSS Kubernetes, Istio and KNative *
>  +1 on above.
>
> 3.*AWS helping with testing infra for this (I assume this will use
> Kubernetes OSS, as opposed to EKS). *
> +1 on above
>
> 4. *Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto,
> Seldon...) are not part of this WG, and need to be managed
> independently.* 
> +1 on above
>
> 4. *A new Manifests repo, no default deployment tool (e.g. kfctl)
> as part of new Control Plane WG*
> This is the only area where I need clarity. So KFCTL and existing
> Manifests repo are going to be managed by Deployment WG
> <https://github.com/kubeflow/community/pull/402>? If yes, should
> it be called something else (e.g. Distributions WG)?
>
> Thanks,
>
> Animesh
>
>
> On Thu, Oct 22, 2020 at 3:31 PM Constantinos Venetsanopoulos
> <cv...@arrikto.com <mailto:cv...@arrikto.com>> wrote:
>
> Hello Yao,
>
> This is great! We'd definitely like that, thank you for your
> support.
>
> Best,
> Constantinos
>
> On 22/10/20 15:16, Work wrote:
> > Hi Constantinos, 
> >
> > This is Yao, from AWS. From AWS side, we’re happy to work
> with you for
> > setting up CI process to test on shared test-infra. There
> are already
> > some POC work ongoing. 
> >
> > Thanks,
> > Yao
> > On Oct 22, 2020, 3:09 PM -0700, Constantinos Venetsanopoulos
> >>> <mailto:jer...@lewi.us <mailto:jer...@lewi.us>>> wrote:
> >>>
> >>>> I would avoid arguing what is the best 3rd party distro
> or try
> >>> to make one the default upstream way, I believe we all benefit
> >>> from open collaboration.
> >>>
> >>> +1
> >>>
> >>> We have always endeavored to be neutral and not endorse one
> >>> distribution over another. I think we should continue to
> push in
> >>> this direction. Users should be presented with a list of
> options
> >>> and choose the one that works best for them.
> >>>
> >>> J
> >>>
> >>> On Thu, Oct 22, 2020 at 12:32 PM Rui Vasconcelos
> >>> <rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>
> >>> <mailto:rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>>> wrote:
> >>>
> >>> Hi everyone!
> >>>
> >>> It seems that a lot of people in the community have a viable
> >>> e2e platform, day-0 and day-2 stories - Arrikto, AgileStacks,
> >>> D2iQ, Canonical, OpenShift Operator - but these are not
> >>> visible to the end-user seeing KF for the first time, and this
> >>> is hurting the community overall.
> >>>
> >>> Raising a few questions :)
> >>>
> >>> * Given that these efforts sit outside of Kubeflow, should
> >>> we create a distro list in the docs to help users find e2e
> >>> deployment methods?
> >>> * Alternatively, should there be a commonplace to host
> >>> distributions?
> >>> * If neither of the above, does kfctl solve the end-user
> >>> problem of simply trying Kubeflow? Can we make it so?
> >>>
> >>> I would avoid arguing what is the best 3rd party distro or try
> >>> to make one the default upstream way, I believe we all benefit
> >>> from open collaboration.
> >>>
> >>> Rui
> >>>
> >>> On Thu, Oct 22, 2020 at 7:13 PM Darryl Ruggles
> >>> <rdar...@gmail.com <mailto:rdar...@gmail.com>
> >>>> <mailto:aron...@gmail.com <mailto:aron...@gmail.com>>> wrote:
> >>>>
> >>>> Hey Josh--
> >>>>
> >>>> Would that mean doing it all in a language (eg
> >>> Python)? Or could
> >>>> K3s be acceptable?
> >>>>
> >>>> On Thu, Oct 22, 2020 at 6:31 AM
> >>> jo...@pattersonconsultingtn.com
> <mailto:jo...@pattersonconsultingtn.com>
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>> <jer...@lewi.us <mailto:jer...@lewi.us>>
> >>>>> <mailto:mtane...@d2iq.com <mailto:mtane...@d2iq.com>>>
> >>>> <mailto:cv...@arrikto.com <mailto:cv...@arrikto.com>>>>
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>>.
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%25252Buns...@googlegroups.com>>>>.
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>>>
> >>>> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com
> <mailto:kubeflow-discuss%252Bunsu...@googlegroups.com>>.
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> >>>>
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Luke Marsden   |   Founder & CEO, MLOps Consulting
> >>>> US  +1 415-231-8523 <tel:(415)%20231-8523>
> >>> <tel:(415)%20231-8523>
> >>>> UK  +44 7791750420 <tel:+44%207791%20750420>
> >>> <tel:+44%207791%20750420>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the
> >>>> Google Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from
> >>>> it, send an email to
> >>>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the Google
> >>>> Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from it,
> >>>> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>>
> >>>> --
> >>>> You received this message because you are subscribed
> >>> to the Google
> >>>> Groups "kubeflow-discuss" group.
> >>>> To unsubscribe from this group and stop receiving
> >>> emails from it, send
> >>>> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>
> >>>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>>.
> >>>> To view this discussion on the web visit
> >>>>
> >>>
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com>>
> >>>
> >>>>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
> >>>
> >>>
> >>> --
> >>> You received this message because you are subscribed to
> >>> the Google Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails
> >>> from it, send an email to
> >>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/d2cdf7eb-a413-4605-8da8-b936ca5f3e67n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>>
> >>>
> >>> --
> >>> Rui Vasconcelos
> >>> Product Manager - AI/ML
> >>> *Canonical | **Ubuntu*
> >>> +351 919154539
> >>> --
> >>> You received this message because you are subscribed to the
> >>> Google Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails from
> >>> it, send an email to
> >>> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmKZ5%2BGp_%2B4DTnpi%2BZ-X1u35wD7tJxcjKDD7JcXBNd6UsA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>> --
> >>> You received this message because you are subscribed to
> the Google
> >>> Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails
> from it,
> >>> send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYVy1sv20vVnTjE6Dtj%3DpP5Rm_kVTOHwg9o7iQoMB%2BuEg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>>
> >>> --
> >>> You received this message because you are subscribed to
> the Google
> >>> Groups "kubeflow-discuss" group.
> >>> To unsubscribe from this group and stop receiving emails
> from it, send
> >>> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>
> >>> <mailto:kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> <https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxZe6U3NFOdFfeZ%3Do5aOT8acV7Y_zy47YMGpHr%3DzWXibDg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> >>
> >> --
> >> You received this message because you are subscribed to the
> Google
> >> Groups "kubeflow-discuss" group.
> >> To unsubscribe from this group and stop receiving emails
> from it,
> >> send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> >> To view this discussion on the web visit
> >>
> https://groups.google.com/d/msgid/kubeflow-discuss/68e41875-66bf-fb35-a2de-d66a2d2482c7%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/68e41875-66bf-fb35-a2de-d66a2d2482c7%40arrikto.com>.
>
> --
> You received this message because you are subscribed to the
> Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discuss%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/bfc408aa-1ab2-6273-1989-ae0b34b2de85%40arrikto.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/bfc408aa-1ab2-6273-1989-ae0b34b2de85%40arrikto.com>.
>

Luke Marsden

unread,
Oct 23, 2020, 2:12:13 AM10/23/20
to Jeremy Lewi, Animesh Singh, Constantinos Venetsanopoulos, Work, David Aronchick, Rui Vasconcelos, kubeflow-discuss
On Fri, 23 Oct 2020, 02:07 Jeremy Lewi, <jer...@lewi.us> wrote:
A core distribution of Kubeflow distributing applications based on core WGs (Notebooks, Training, AutoML, Serving and Pipelines) on vanilla OSS Kubernetes, Istio and KNative 
Vendor specific distributions (e.g. AWS, Google, IBM, Arrikto, Seldon...) are not part of this WG, and need to be managed independently. 

The idea of a "core"/default/official distribution seems problematic. Is there a reason we can't treat all distributions uniformly?

All distributions are opinionated. Whether you are using only OSS components or not; every distribution represents a set of opinionated decisions.

Anything which creates a perception that one distribution is endorsed by Kubeflow over others seems inherently problematic to me.
A WG says Kubeflow is responsible for this distribution and that seems problematic to me. 

Why do we need to create WGs to sponsor distributions? This just seems like an invitation for conflict as various groups agitate to advance their agenda by getting their opinions upstreamed.
Adopting only OSS tech doesn't preempt this (emacs vs. vim, helm vs kustomize, etc...)

If folks want to collaborate on a distribution, why don't they just pick a name spin up a GitHub repo and invite interested parties to collaborate? If we adopt a policy of no distributions within Kubeflow then we carve out a clear space for an ecosystem of distributions.


Think about this from the user's perspective though. They hear about this cool thing called kubeflow, and they want to try it out. In the future you're describing, they're presented with a complex matrix of different opinionated distributions and a fragmented set of tools to spin up the thing, all of which are probably  broken in slightly different ways. They're forced to make difficult decisions before they can even get started! Before they can see _any_ value. This is a really bad UX. It's really bad for adoption for the Kubeflow project. Do we want the project to be successful and have many users?

This is exactly what the situation was with Kubernetes before kubeadm. And kubeadm solved a huge pent up demand to make kube easier to install in an officially supported way. Arguably kubeflow's job is much easier here, setting up a stock OSS install of kubeflow is much easier than installing kubernetes on Linux!

So despite my affiliations, honestly for the good of the project I support the arrikto proposal :)

And btw another thing kubeadm did was as well as creating an end user tool with slick UX, also provided a library which made other installation tools' job much easier. Minikube is now a wrapper around kubeadm for example. It improved consistency and decreased complexity in the tooling.

Hope this perspective is helpful and constructive. Happy to talk more about this analogous experience if it's helpful.

Cheers,
Luke

Rui Vasconcelos

unread,
Oct 23, 2020, 6:06:47 AM10/23/20
to Luke Marsden, Jeremy Lewi, Animesh Singh, Constantinos Venetsanopoulos, Work, David Aronchick, kubeflow-discuss
+1 on clean UX, and if kfctl is the only way +1 on that too.

I do see one governance issue with that though. If kfctl is controlled by a subset of the community that also produces distros, it is hard to enforce that this work is not meshed. One example of this is the OpenShift KF operator being embedded into kfctl -> #411

This goes against Constantinos' point that:
> almost everyone in the community agrees that this is not really an opinionated distro.

In addition, by having some distro vendors working upstream and others in independent repos, the former can state that they are the only ones contributing, and hence should be the ones to make decisions rather than the broader community.

How could we enforce that kfctl stays fully agnostic?
--
Rui Vasconcelos
Product Manager - AI/ML
Canonical | Ubuntu
+351 919154539

Johnu

unread,
Oct 23, 2020, 8:02:05 AM10/23/20
to Rui Vasconcelos, Luke Marsden, Jeremy Lewi, Animesh Singh, Constantinos Venetsanopoulos, Work, David Aronchick, kubeflow-discuss

Great to see discussion on this topic.  


+1 to the argument for maintaining a minimalistic base reference architecture for users to try out Kubeflow and various components.   Vendor neutral config like kfctl_k8s_istio (https://www.kubeflow.org/docs/started/k8s/overview/) for vanilla KF deployments is a good example.   Given the fact that there are limitless customizations possible for a ML platform, maintaining opinionated configs beyond the reference ones inside Kubeflow looks unsustainable.  Is the new proposal different from current kfctl_k8s_istio ? 



Johnu


--
You received this message because you are subscribed to the Google Groups "kubeflow-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.

Maxim Veksler

unread,
Oct 23, 2020, 9:13:40 AM10/23/20
to kubeflow-discuss
Hello everyone, great discussion.

We are a startup, we use ML to sort seeds. Users of Kubeflow since 0.7.
I'd be honored to add my 2 cents. 

We chose Kubeflow after doing some solutions research. The decision that moved us to choose Kubeflow was the impression that the platform will act as a comprehensive solution where best practices will emerge as a result of a joined learning done in open source by the community. This strategy has not yet paid off in terms of effort we've put to get Kubeflow to production readiness. However, we do like the tool and continue to use and integrate with it both at the API and at the metadata (currently going directly to the DB) level as it serves as a reasonable foundation for our current needs, in the hope that future benefits will emerge.

The components we use from Kubeflow arsenal today are: 
  1. Pipelines
  2. Experiment + Runs metadata management (accessed from Kubeflow pipelines frontend, and indirectly by our platform integrated over REST API)

Since we've gained some experience with the tool, and explored its options and integrations I think that it might be doing a bit too much in terms of the spread of approaches. For us, as a greenfield user, it would have been better if the project worked in a certain way and recommended it while listing the alternatives with a maturity tag attached to each (maturity could probably be defined by something like test coverage + release cadence + # of undiscovered critical bugs. i.e. Accelerate KPI's)

What would be great IMHO for the Kubeflow project is to allow for modularity where the core "stable" solution would be the best-practiced approach to ML, while the rest will be governed by a similar structure to Apache foundations policy of incubation projects that form a community around them.

A "best praciteced" for us includes (note: heavily biased to our needs)
  • dataset engineering (=heartexlabs/label-studio+dvc/quilt)
  • training(=pipelines+mlops+automl)
  • exploring(=~wandb+datadog+artifacts+notebook)
  • and managing(=allegroai/trains)
* marked in () are approximations offered for clarity and explainability. None is an endorsement or recommendation.



Respectfully,

On Wed, Oct 7, 2020 at 7:17 PM Jeremy Lewi <jer...@lewi.us> wrote:
Hi Folks,

Over the past couple of months we've been working hard to divide Kubeflow into working groups responsible for different KF applications. 

We now have 5 different work groups (listed here) that own all the major applications.

There is an open question about what to do about some of the remaining applications/pieces that don't fit neatly into these work groups. These include

Auth/Dex/Profile Controller
Reference Architectures / generic KFDefs

We've been debating what to do at the recent community meetings. I wanted to start a thread to give everyone an opportunity to chime in.

To summarize it looks like there are two proposals on the table

1. Spin up appropriate WGs to own and maintain these applications going forward
2. KF stops assuming responsibility for bundling and deploying all the applications
  • Each application should be installable on its own with that responsibility assumed by the applications owners and/or WG
  • Vendors would be responsible for composing the applications into distributions and providing glue (e.g. Dex or middleware like ISTIO) at their discretion.
#2 raises the question what ties the various applications into a cohesive platform?

One possibility is that pipelines & metadata not deployment is what binds together the applications into a cohesive platform.

This idea is something we first discussed at our 2019 summit. All Kubeflow applications would follow some conventions that enable:
  1. using them in pipelines
  2. consistently logging metadata
Adhering to these conventions is what would distinguish a KF application from a non KF application.  

Thoughts?
J





--
You received this message because you are subscribed to the Google Groups "kubeflow-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.


--

Jeremy Lewi

unread,
Oct 23, 2020, 10:42:55 AM10/23/20
to Maxim Veksler, kubeflow-discuss
There's a lot of great discussion in this thread (I particularly appreciate all the feedback from users).

Constantinos regarding the specific WG proposal; let's follow Kubeflow's WG process, I think there are two possible paths here

1. Start a new thread focused on scoping the WG and get conditional preapproval
2. Skip step 1 and turn it into a PR using the appropriate template

J


 

Jeremy Lewi

unread,
Oct 23, 2020, 11:49:38 AM10/23/20
to Maxim Veksler, kubeflow-discuss
+1 to the argument for maintaining a minimalistic base reference architecture for users to try out Kubeflow and various components.  

If minimal is what we are going for why not start with just KFP?  One application. No middleware (ISTIO, DEX, KNative, Cloud Infra, etc...)
Just "kubectl apply  -f "

If you believe(as I do) that pipelines/metadata is the core that ties all of these different applications into a cohesive platform
then it seems logical that you start by installing KFP/medata and then tailor the platform to an org's preferences by installing the applications relevant to that org.

J

David Aronchick

unread,
Oct 23, 2020, 12:15:53 PM10/23/20
to Jeremy Lewi, Maxim Veksler, kubeflow-discuss
I don't disagree with "starting with KFP", but I think the problem is "what do you do with it".

That's why a small subset of core functions (e.g. Juptyer, Kale, TFJob/PyTorchJob, Katib & KFServing) should also be part of this core tested platform - even if some organizations disable these right out of the box.

Maxim Veksler

unread,
Oct 23, 2020, 12:42:18 PM10/23/20
to Jeremy Lewi, kubeflow-discuss
Hi Jeremy,

I'm not sure this question was directed to me, as I did not take part in the earlier discussions.

If I could offer an opinion, looking at this from an engineering/architecture perspective could "pipelines/metadata is the core that ties all of these different applications into a cohesive platform" be restated as "compliance to an API"? 

If so then Kubeflow "core" would become the "pipelines and metadata" and provide the Data Modeling, infrastructure & Persistency layers to enable all other loosely coupled components to speak one with the other. This way the goal of the project becomes to identify, refine, generalize and set these integration "specs" and also push for the implementation (reference impl.) of each, either by wrapping an existing solution (TF, PyTorch, Triton Inference) or creating and building the missing blocks.

This is somewhat similar to how elastic-search is able to operate as the core API + Modeling + Indexing as the core, allowing services such as log aggregations, monitoring & graphing on top.

As a user of Kubeflow, If we could have started with a simpler install of only pipelines without all the other parts of the system it would simplify things for us and ease the introduction of the tool. 

However, I imagine that the value of Kubeflow is offering a solution to the MLOps and I'd perhaps suggest even to the ResearchOps side of things, both of which are not a solved problem just yet.

Constantinos Venetsanopoulos

unread,
Oct 23, 2020, 2:47:16 PM10/23/20
to Jeremy Lewi, Maxim Veksler, kubeflow-discuss
Hello Jeremy,

I agree. Let us start working on Mon to turn this into a proper PR.

And by then, let me come back with some final comments in this thread,
since I see more replies today and also will start answering to the
comments in the doc.

Thank you,
Constantinos

On 23/10/20 07:42, Jeremy Lewi wrote:
> There's a lot of great discussion in this thread (I particularly
> appreciate all the feedback from users).
>
> Constantinos regarding the specific WG proposal
> <https://docs.google.com/document/d/1BpR4RTp4m2GO_r-c3_xZ-qGnCfvQdqiRh4V0cFUSBqA/edit#heading=h.iat9otio1a62>;
> let's follow Kubeflow's WG process
> <https://github.com/kubeflow/community/blob/master/wgs/wg-lifecycle.md>,
> I think there are two possible paths here
>
> 1. Start a new thread focused on scoping the WG and get conditional
> preapproval
> 2. Skip step 1 and turn it into a PR using the appropriate template
>
> J
>
>
>  
>
> On Fri, Oct 23, 2020 at 6:13 AM Maxim Veksler <m...@seed-x.com
> <mailto:m...@seed-x.com>> wrote:
>
> Hello everyone, great discussion.
>
> We are a startup, we use ML to sort seeds. Users of Kubeflow since
> 0.7.
> I'd be honored to add my 2 cents. 
>
> We chose Kubeflow after doing some solutions research
> <https://docs.google.com/document/d/1jHLTkR9WSaOQ4uQRfF9_udGntFg0MFIaPJtJOo2qLWg/edit?usp=sharing>.
> The decision that moved us to choose Kubeflow was the impression
> that the platform will act as a comprehensive solution where best
> practices will emerge as a result of a joined learning done in
> open source by the community. This strategy has not yet paid off
> in terms of effort we've put to get Kubeflow to production
> readiness. However, we do like the tool and continue to use and
> integrate with it both at the API and at the metadata (currently
> going directly to the DB) level as it serves as
> a reasonable foundation for our current needs, in the hope that
> future benefits will emerge.
>
> The components we use from Kubeflow arsenal today are: 
>
> 1. Pipelines
> 2. Experiment + Runs metadata management (accessed from
> Kubeflow pipelines frontend, and indirectly by our platform
> integrated over REST API)
> 3.
>
> Since we've gained some experience with the tool, and explored its
> options and integrations I think that it might be doing a bit too
> much in terms of the spread of approaches. For us, as a greenfield
> user, it would have been better if the project worked in a
> certain way and recommended it while listing the alternatives with
> a maturity tag attached to each (maturity could probably be
> defined by something like test coverage + release cadence + # of
> undiscovered critical bugs. i.e. Accelerate KPI's
> <https://www.thoughtworks.com/radar/techniques/four-key-metrics>)
>
> What would be great IMHO for the Kubeflow project is to allow for
> modularity where the core "stable" solution would be the
> best-practiced approach to ML, while the rest will be governed by
> a similar structure to Apache foundations policy of incubation
> projects that form a community around them.
>
> A "best praciteced" for us includes (note: _/heavily biased to our
> needs/_)
>
> * dataset engineering (=heartexlabs/label-studio+dvc/quilt)
> * training(=pipelines+mlops+automl)
> * exploring(=~wandb+datadog+artifacts+notebook)
> * and managing(=allegroai/trains)
>
> /* marked in () are approximations offered for clarity and
> explainability. None is an endorsement or recommendation./
>
>
>
> Respectfully,
>
> On Wed, Oct 7, 2020 at 7:17 PM Jeremy Lewi <jer...@lewi.us
> <mailto:jer...@lewi.us>> wrote:
>
> Hi Folks,
>
> Over the past couple of months we've been working hard to
> divide Kubeflow into working groups responsible for different
> KF applications. 
>
> We now have 5 different work groups (listed here
> <https://github.com/kubeflow/community/blob/master/wg-list.md>)
> that own all the major applications.
>
> There is an open question about what to do about some of the
> remaining applications/pieces that don't fit neatly into these
> work groups. These include
>
> central dashboard -
> (https://github.com/kubeflow/community/issues/380
> <https://github.com/kubeflow/community/issues/380>)
> pod defaults controller -
> (https://github.com/kubeflow/community/issues/381
> <https://github.com/kubeflow/community/pull/402>)
> Auth/Dex/Profile Controller
> Reference Architectures / generic KFDefs
>
> We've been debating what to do at the recent community
> meetings. I wanted to start a thread to give everyone an
> opportunity to chime in.
>
> To summarize it looks like there are two proposals on the table
>
> 1. Spin up appropriate WGs to own and maintain these
> applications going forward
> 2. KF stops assuming responsibility for bundling and deploying
> all the applications
>
> * Each application should be installable on its own with
> that responsibility assumed by the applications owners
> and/or WG
> * Vendors would be responsible for composing the
> applications into distributions and providing glue (e.g.
> Dex or middleware like ISTIO) at their discretion.
>
> #2 raises the question what ties the various applications into
> a cohesive platform?
>
> One possibility is that pipelines & metadata not deployment is
> what binds together the applications into a cohesive platform.
>
> This idea is something we first discussed at our 2019 summit
> <https://docs.google.com/presentation/d/13a3shc98F-G779tnNFXb4ZYfhniInI--NtUanIivkOM/edit#slide=id.g51ce843281_2_1>.
> All Kubeflow applications would follow some conventions that
> enable:
>
> 1. using them in pipelines
> 2. consistently logging metadata
>
> Adhering to these conventions is what would distinguish a KF
> application from a non KF application.  
>
> Thoughts?
> J
>
>
>
>
>
> --
> You received this message because you are subscribed to the
> Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYimdfLrw51fs9fh6yrYn2_sVqmcpQH9uDni0da_QSVyg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CABW14FNYN1QX%2BEKsrSLsLi4Bhk%3DPOkb9KznkmcL%3DkGTVR-yduw%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CABW14FNYN1QX%2BEKsrSLsLi4Bhk%3DPOkb9KznkmcL%3DkGTVR-yduw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYmwzrJs-fe4LPy0-8qeZX6WkngoVaCSoUsUOE%3DUcSAtg%40mail.gmail.com
> <https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiYmwzrJs-fe4LPy0-8qeZX6WkngoVaCSoUsUOE%3DUcSAtg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Jeremy Lewi

unread,
Oct 23, 2020, 2:53:34 PM10/23/20
to Maxim Veksler, kubeflow-discuss
Maxim,

I really like how you've laid out the vision. I believe that is the right direction for Kubeflow although it will take us some time to get there. 

If you go back to one of the long buried posts I shared a link to a doc with some initial ideas for what it might look like for an application to be Kubeflow Native

J

j...@spotify.com

unread,
Oct 23, 2020, 7:04:22 PM10/23/20
to kubeflow-discuss
Just to add another user perspective from Spotify as a large user of KF(TL;DR: very much in agreement with what Maxim posted): 

Pipelines and the metadata elements have been the most important/impactful aspects of the project and a focus on improving that out-of-the-box experience could go a long way towards onboarding new users and growing a larger community. Metadata as the glue that binds together the platform and Pipelines as the primary experience that demonstrates the power of the system resonates a lot with how our own ml platform building and usage of Kubeflow looks. We love the optionality that Kubeflow gives us and that approach would seem to make it even easier to onboard additional elements of the Kubeflow ecosystem. It could also make it easier to contribute products back to the ecosystem for users like us. 

This approach would allow for a good evolution of the product too as over time more and more elements become part of the core offering (monitoring, feature store, improved overview UI, etc), and allow the project to grow in value for users like Spotify. It would also seem to be very supportive of vendors flavoring their own experience on top of the core and offering proprietary elements and experiences that save users from making decisions on the various KF options.

I know the conversation has moved a bit beyond that original question (and I can't say we quite follow all that discussion) but thought it might be good to +1 Maxin's user perspective post!

Cheers!
Josh
(ML Platform @ Spotify)

Ajay

unread,
Oct 23, 2020, 9:27:51 PM10/23/20
to j...@spotify.com, kubeflow-discuss
+1 to Maxim's suggestion here as well. I like the idea of Pipelines & Metadata exporting APIs, data models, specs etc to provide integration points for various other services to bring additional/optional value-add to the core CUJs. 

- Ajay


--
You received this message because you are subscribed to the Google Groups "kubeflow-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubeflow-discu...@googlegroups.com.

Constantinos Venetsanopoulos

unread,
Oct 23, 2020, 9:49:13 PM10/23/20
to Rui Vasconcelos, Luke Marsden, Jeremy Lewi, Animesh Singh, Work, David Aronchick, kubeflow-discuss
Hello Rui,

Please note that in the proposal we state that we are NOT going to be
maintaining kfctl, because kfctl is an opinionated deployment tool and
not everyone uses it. We will maintain a base set of manifests, which
people can use as is, or even create overlays that cater to kfctl.

Our instructions for the base deployment will not include the use of kfctl.

Thank you,
Constantinos

On 23/10/20 03:06, Rui Vasconcelos wrote:
> +1 on clean UX, and if kfctl is the only way +1 on that too.
>
> I do see one governance issue with that though. If kfctl is controlled
> by a subset of the community that also produces distros, it is hard to
> enforce that this work is not meshed. One example of this is the
> OpenShift KF operator being embedded into kfctl -> #411
> <https://github.com/kubeflow/kfctl/pull/411>
>
> This goes against Constantinos' point that:
> > almost everyone in the community agrees that this is not really an
> opinionated distro.
>
> In addition, by having some distro vendors working upstream and others
> in independent repos, the former can state that they are the only ones
> contributing, and hence should be the ones to make decisions rather
> than the broader community.
>
> How could we enforce that kfctl stays fully agnostic?
>
> On Fri, Oct 23, 2020 at 7:12 AM Luke Marsden <lu...@mlops.consulting>
> wrote:
>
> On Fri, 23 Oct 2020, 02:07 Jeremy Lewi, <jer...@lewi.us
> <mailto:jer...@lewi.us>> wrote:
>
> > *A core distribution of Kubeflow distributing applications
> based on core WGs (Notebooks, Training, AutoML, Serving and
> Pipelines) on vanilla OSS Kubernetes, Istio and KNative *
> *> **Vendor specific distributions (e.g. AWS, Google, IBM,
> Arrikto, Seldon...) are not part of this WG, and need to be
> managed independently.* 
> 1. *Each application WG producing a Pipeline Component and
> Metadata to be plugged into KFP with every release*
> +1 on above
>
> 2. *A core distribution of Kubeflow distributing
> applications based on core WGs (Notebooks, Training,
> AutoML, Serving and Pipelines) on vanilla OSS Kubernetes,
> Istio and KNative *
>  +1 on above.
>
> 3.*AWS helping with testing infra for this (I assume this
> will use Kubernetes OSS, as opposed to EKS). *
> +1 on above
>
> 4. *Vendor specific distributions (e.g. AWS, Google, IBM,
> Arrikto, Seldon...) are not part of this WG, and need to
> be managed independently.* 
> +1 on above
>
> 4. *A new Manifests repo, no default deployment tool (e.g.
> kfctl) as part of new Control Plane WG*
> This is the only area where I need clarity. So KFCTL and
> existing Manifests repo are going to be managed by
> Deployment WG
> <https://github.com/kubeflow/community/pull/402>? If yes,
> should it be called something else (e.g. Distributions WG)?
>
> Thanks,
>
> Animesh
>
>
> On Thu, Oct 22, 2020 at 3:31 PM Constantinos
> Venetsanopoulos <cv...@arrikto.com
> <mailto:cv...@arrikto.com>> wrote:
>
> Hello Yao,
>
> This is great! We'd definitely like that, thank you
> for your support.
>
> Best,
> Constantinos
>
> On 22/10/20 15:16, Work wrote:
> > Hi Constantinos, 
> >
> > This is Yao, from AWS. From AWS side, we’re happy to
> work with you for
> > setting up CI process to test on shared test-infra.
> There are already
> > some POC work ongoing. 
> >
> > Thanks,
> > Yao
> > On Oct 22, 2020, 3:09 PM -0700, Constantinos
> Venetsanopoulos
> >>> <mailto:jer...@lewi.us <mailto:jer...@lewi.us>>>
> wrote:
> >>>
> >>>> I would avoid arguing what is the best 3rd party
> distro or try
> >>> to make one the default upstream way, I believe we
> all benefit
> >>> from open collaboration.
> >>>
> >>> +1
> >>>
> >>> We have always endeavored to be neutral and not
> endorse one
> >>> distribution over another. I think we should
> continue to push in
> >>> this direction. Users should be presented with a
> list of options
> >>> and choose the one that works best for them.
> >>>
> >>> J
> >>>
> >>> On Thu, Oct 22, 2020 at 12:32 PM Rui Vasconcelos
> >>> <rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>
> >>> <mailto:rui.vas...@canonical.com
> <mailto:rdar...@gmail.com
> >>>> <mailto:aron...@gmail.com
> <mailto:aron...@gmail.com>>> wrote:
> >>>>
> >>>> Hey Josh--
> >>>>
> >>>> Would that mean doing it all in a language (eg
> >>> Python)? Or could
> >>>> K3s be acceptable?
> >>>>
> >>>> On Thu, Oct 22, 2020 at 6:31 AM
> >>> jo...@pattersonconsultingtn.com
> <mailto:jo...@pattersonconsultingtn.com>
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>> <jer...@lewi.us <mailto:jer...@lewi.us>>
> <mailto:mathew...@gmail.com>> wrote:
> >>>>
> >>>> Hey William,
> >>>>
> >>>> If my proposal were to be accepted by Kubeflow,
> >>>> nothing is to prevent Pipelines from being made
> >>>> available outside of the full Kubeflow platform,
> >>>> (with the caveat that Kubeflow Pipelines would
> >>>> need to rebrand itself to not use the word
> >>>> Kubeflow, as Kubeflow is the entire platform).
> >>>>
> >>>> However, a better solution for your use-case would
> >>>> be to install the *generic distribution *and only
> >>>> select the Pipelines component. 
> >>>>
> >>>> Naturally, the generic distribution would be
> >>>> modular, with only some components required, like
> >>>> for example, the Central Dashboard (assuming you
> >>>> wanted any form of WebUI).
> >>>>
> >>>> On Tuesday, 13 October 2020 at 6:59:10 pm UTC+11
> >>>> etern...@gmail.com <mailto:etern...@gmail.com> wrote:
> >>>>
> >>>> Hi Mathew
> >>>>
> >>>> I have a bit of issue with the following
> >>>> statement. 
> >>>>
> >>>> * Every UI exposed by Kubeflow should
> >>>> be* only accessible* through the secure
> >>>> central dashboard
> >>>>
> >>>> Mostly because as mentioned previously, there
> >>>> are 2 types of users. Those that use only some
> >>>> components of kubeflow, and those who view
> >>>> kubeflow as a whole.
> >>>>
> >>>> For e.g., I would want to be able to use kfp
> >>>> independently from kubeflow per se - esp when
> >>>> I am running a very light weight distribution
> >>>> for a small team.
> >>>>
> >>>> The part I like about kubeflow is that it is
> >>>> easy for me to integrate third party (or
> >>>> in-house) components into the kubeflow
> >>>> "framework".
> >>>>
> >>>> Although I would hope central dashboard can be
> >>>> modified to be more composible so that I can
> >>>> customize w/o maintaining a fork.
> >>>>
> >>>>
> >>>> On Tue, 13 Oct 2020, 8:21 am Mathew Wicks,
> >>>> <mathew...@gmail.com
> >>>>> <mailto:mtane...@d2iq.com
> *Canonical | **Ubuntu*
> +351 919154539

Constantinos Venetsanopoulos

unread,
Oct 23, 2020, 9:51:31 PM10/23/20
to Johnu, Rui Vasconcelos, Luke Marsden, Jeremy Lewi, Animesh Singh, Work, David Aronchick, kubeflow-discuss
Hello Johnu,

Thank you very much for your feedback. Note that in our proposal we
explicitly note that we will not depend on kfctl, thus we will not
maintain it, or have instructions that use it. If someone wants to use
kfctl as their tool of choice they will have to maintain it separately
and create overlays on the base manifests.

Thank you,
Constantinos

On 23/10/20 05:01, Johnu wrote:
>
> Great to see discussion on this topic.  
>
>
> +1 to the argument for maintaining a minimalistic base reference
> architecture for users to try out Kubeflow and various components.  
> Vendor neutral config like kfctl_k8s_istio
> (https://www.kubeflow.org/docs/started/k8s/overview/
> <https://www.kubeflow.org/docs/started/k8s/overview/>) for vanilla KF
> deployments is a good example.   Given the fact that there are
> limitless customizations possible for a ML platform, maintaining
> opinionated configs beyond the reference ones inside Kubeflow looks
> unsustainable.  Is the new proposal different from current
> kfctl_k8s_istio ? 
>
>
>
> Johnu
>
>
> On Fri, Oct 23, 2020 at 3:36 PM Rui Vasconcelos
> <rui.vas...@canonical.com <mailto:rui.vas...@canonical.com>>
> wrote:
>
> +1 on clean UX, and if kfctl is the only way +1 on that too.
>
> I do see one governance issue with that though. If kfctl is
> controlled by a subset of the community that also produces
> distros, it is hard to enforce that this work is not meshed.
> One example of this is the OpenShift KF operator being embedded
> into kfctl -> #411 <https://github.com/kubeflow/kfctl/pull/411>
>
> This goes against Constantinos' point that:
> > almost everyone in the community agrees that this is not really
> an opinionated distro.
>
> In addition, by having some distro vendors working upstream and
> others in independent repos, the former can state that they are
> the only ones contributing, and hence should be the ones to make
> decisions rather than the broader community.
>
> How could we enforce that kfctl stays fully agnostic?
>
> On Fri, Oct 23, 2020 at 7:12 AM Luke Marsden
> <lu...@mlops.consulting> wrote:
>
> On Fri, 23 Oct 2020, 02:07 Jeremy Lewi, <jer...@lewi.us
> <mailto:jer...@lewi.us>> wrote:
>
> > *A core distribution of Kubeflow distributing
> applications based on core WGs (Notebooks, Training,
> AutoML, Serving and Pipelines) on vanilla OSS Kubernetes,
> Istio and KNative *
> *> **Vendor specific distributions (e.g. AWS, Google, IBM,
> Arrikto, Seldon...) are not part of this WG, and need to
> be managed independently.* 
> <animat...@gmail.com <mailto:animat...@gmail.com>>
> wrote:
>
> Thanks all.
>
> This thread went in many directions, so summarizing
> the thread based on my understanding so far
>
> 1. *Each application WG producing a Pipeline Component
> and Metadata to be plugged into KFP with every release*
> +1 on above
>
> 2. *A core distribution of Kubeflow distributing
> applications based on core WGs (Notebooks, Training,
> AutoML, Serving and Pipelines) on vanilla OSS
> Kubernetes, Istio and KNative *
>  +1 on above.
>
> 3.*AWS helping with testing infra for this (I assume
> this will use Kubernetes OSS, as opposed to EKS). *
> +1 on above
>
> 4. *Vendor specific distributions (e.g. AWS, Google,
> IBM, Arrikto, Seldon...) are not part of this WG, and
> need to be managed independently.* 
> +1 on above
>
> 4. *A new Manifests repo, no default deployment tool
> (e.g. kfctl) as part of new Control Plane WG*
> This is the only area where I need clarity. So KFCTL
> and existing Manifests repo are going to be managed by
> Deployment WG
> <https://github.com/kubeflow/community/pull/402>? If
> yes, should it be called something else (e.g.
> Distributions WG)?
>
> Thanks,
>
> Animesh
>
>
> On Thu, Oct 22, 2020 at 3:31 PM Constantinos
> Venetsanopoulos <cv...@arrikto.com
> <mailto:cv...@arrikto.com>> wrote:
>
> Hello Yao,
>
> This is great! We'd definitely like that, thank
> you for your support.
>
> Best,
> Constantinos
>
> On 22/10/20 15:16, Work wrote:
> > Hi Constantinos, 
> >
> > This is Yao, from AWS. From AWS side, we’re
> happy to work with you for
> > setting up CI process to test on shared
> test-infra. There are already
> > some POC work ongoing. 
> >
> > Thanks,
> > Yao
> > On Oct 22, 2020, 3:09 PM -0700, Constantinos
> Venetsanopoulos
> >>> <mailto:jer...@lewi.us
> <mailto:jer...@lewi.us>>> wrote:
> >>>
> >>>> I would avoid arguing what is the best 3rd
> party distro or try
> >>> to make one the default upstream way, I
> believe we all benefit
> >>> from open collaboration.
> >>>
> >>> +1
> >>>
> >>> We have always endeavored to be neutral and
> not endorse one
> >>> distribution over another. I think we should
> continue to push in
> >>> this direction. Users should be presented with
> a list of options
> >>> and choose the one that works best for them.
> >>>
> >>> J
> >>>
> >>> On Thu, Oct 22, 2020 at 12:32 PM Rui Vasconcelos
> >>> <rui.vas...@canonical.com
> <mailto:rui.vas...@canonical.com>
> >>> <mailto:rui.vas...@canonical.com
> <mailto:rdar...@gmail.com
> >>>> <mailto:aron...@gmail.com
> <mailto:aron...@gmail.com>>> wrote:
> >>>>
> >>>> Hey Josh--
> >>>>
> >>>> Would that mean doing it all in a language (eg
> >>> Python)? Or could
> >>>> K3s be acceptable?
> >>>>
> >>>> On Thu, Oct 22, 2020 at 6:31 AM
> >>> jo...@pattersonconsultingtn.com
> <mailto:jo...@pattersonconsultingtn.com>
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>>> <mailto:jo...@pattersonconsultingtn.com
> >>> <jer...@lewi.us <mailto:jer...@lewi.us>>
> <mailto:mathew...@gmail.com>> wrote:
> >>>>
> >>>> Hey William,
> >>>>
> >>>> If my proposal were to be accepted by Kubeflow,
> >>>> nothing is to prevent Pipelines from being made
> >>>> available outside of the full Kubeflow platform,
> >>>> (with the caveat that Kubeflow Pipelines would
> >>>> need to rebrand itself to not use the word
> >>>> Kubeflow, as Kubeflow is the entire platform).
> >>>>
> >>>> However, a better solution for your use-case
> would
> >>>> be to install the *generic distribution *and only
> >>>> select the Pipelines component. 
> >>>>
> >>>> Naturally, the generic distribution would be
> >>>> modular, with only some components required, like
> >>>> for example, the Central Dashboard (assuming you
> >>>> wanted any form of WebUI).
> >>>>
> >>>> On Tuesday, 13 October 2020 at 6:59:10 pm UTC+11
> >>>> etern...@gmail.com
> <mailto:etern...@gmail.com> wrote:
> >>>>
> >>>> Hi Mathew
> >>>>
> >>>> I have a bit of issue with the following
> >>>> statement. 
> >>>>
> >>>> * Every UI exposed by Kubeflow should
> >>>> be* only accessible* through the secure
> >>>> central dashboard
> >>>>
> >>>> Mostly because as mentioned previously, there
> >>>> are 2 types of users. Those that use only some
> >>>> components of kubeflow, and those who view
> >>>> kubeflow as a whole.
> >>>>
> >>>> For e.g., I would want to be able to use kfp
> >>>> independently from kubeflow per se - esp when
> >>>> I am running a very light weight distribution
> >>>> for a small team.
> >>>>
> >>>> The part I like about kubeflow is that it is
> >>>> easy for me to integrate third party (or
> >>>> in-house) components into the kubeflow
> >>>> "framework".
> >>>>
> >>>> Although I would hope central dashboard can be
> >>>> modified to be more composible so that I can
> >>>> customize w/o maintaining a fork.
> >>>>
> >>>>
> >>>> On Tue, 13 Oct 2020, 8:21 am Mathew Wicks,
> >>>> <mathew...@gmail.com
> >>>>> <mailto:mtane...@d2iq.com
> *Canonical | **Ubuntu*
> +351 919154539
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kubeflow-discu...@googlegroups.com
> <mailto:kubeflow-discu...@googlegroups.com>.
> <https://groups.google.com/d/msgid/kubeflow-discuss/CA%2B%3DSMmJrs9TJJYp9Le3iRToxWdC0wDTROdDu1p7vyzuEN5ZVZA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

Mathew Wicks

unread,
Oct 27, 2020, 7:25:54 PM10/27/20
to kubeflow-discuss
Please comment on this proposal: https://github.com/kubeflow/community/pull/434

The proposal is to move vendor Kubeflow distributions outside of the Kubeflow repositories, however, there is still a discussion on whether a "generic" distribution of Kubeflow should remain.

NOTE: 
   - this proposal IS NOT about using metadata/pipelines as a "glue" between Kubeflow components (that is pretty clearly the correct path)
   - this proposal IS about ensuring we are not blocked from updating/releasing Kubeflow by the growing number of distributions
   - this proposal IS about ensuring that no single vendor controls Kubeflow
Reply all
Reply to author
Forward
0 new messages