Hello Jeremy,
In general I agree with most of the comments on the AI/ML focus.
Regarding distros:
We are not discussing about an opinionated distribution here. MiniKF is
indeed an opinionated distribution, since it contains customizations to
Kubeflow that haven't yet been upstreamed, and Arrikto's Rok, which is
proprietary. The goal for this effort is to just provide a way to deploy
the current open source components of Kubeflow and stitch them together
as happened in the past. Essentially replicate mostly Google's previous
effort, which everyone in the community loved. This is meant to be a
base to help people create distros, as Animesh commented in the previous
community meeting. This way, they won't have to re-invent, and re-test
everything every time. So, if the community agrees that this is useful,
I think it should definitely be part of Kubeflow.
Best,
Constantinos
On 22/10/20 09:22, Jeremy Lewi wrote:
> I think everyone agrees that ease of deployment and a great day 0 as
> well as day 2 experience is important.
>
> The question IMO: Is the focus on deployment distracting us as
> community from AI/ML?
>
> I think it is. I would much rather we as a community spend our time
> debating questions like
>
> * How should applications emit metadata consistently
> * What should the schemas be for models / datasets/ pipelines etc...
> * How do you design custom resources so they are easily orchestrable
> with pipelines.
> * How do you monitor model performance
>
> Then debating questions like
>
> * What middle ware should be included (ISTIO? Dex? minio?....)?
> * What declarative application management tool chain should be used
> <mailto:
jo...@pattersonconsultingtn.com>
> <
jo...@pattersonconsultingtn.com
> <mailto:
jo...@pattersonconsultingtn.com>> wrote:
>
> I'd like to echo what Luke is saying here; for adoption
> purposes, the "hello world"-aspect of Kubeflow needs to be as
> simple as possible.
>
> right now that's not the case.
>
> Towards that end: If pipelines and metadata are to be
> foundational to Kubeflow going forward, then I'd propose a
> command-line script included with kubeflow that let's a data
> scientist quickly mock up a pipeline and run it locally
> (without installing KF or K8s).
>
> That way, pipelines can gain adoption quickly for the python /
> pandas crowd as an easy workflow engine --- that has the
> upside of the full Kubeflow platform behind it the moment they
> need the full deal.
>
> JP
>
> Patterson Consulting
>
www.pattersonconsultingtn.com
> <
http://www.pattersonconsultingtn.com>
>
> On Thursday, October 15, 2020 at 3:44:13 PM UTC-4 Luke Marsden
> wrote:
>
> Metadata consistency is great, but I personally think that
> ease of deployment is a /much/ more fundamental need for
> the success of the project.
>
> If users can't easily try out the platform and deploy the
> platform in production, all the metadata consistency in
> the world won't help the fact that no one can use it. :)
>
> Developing kubeadm
> <
https://kubernetes.io/blog/2016/09/how-we-made-kubernetes-easy-to-install/>
> helped adoption of Kubernetes, which was way too hard to
> install before that. Let's not have kubeflow go backwards
> on that point. We can't rely on vendors to distribute
> kubeflow.
>
> Just my 2c.
>
> Cheers,
> Luke
>
> On Thu, 15 Oct 2020 at 14:34, Jeremy Lewi <
jer...@lewi.us>
> wrote:
>
> I expanded on this a bit more
> in:
http://bit.ly/kna-proposal
> <
http://bit.ly/kna-proposal>
>
> I agree that we need a cohesive platform. I think that
> cohesion should come from metadata.
>
> For example, no matter how a user trains a model (in a
> notebook, via kubectl, as part of a pipeline) metadata
> about that should be logged and viewable consistently.
>
> To demo the cohesiveness of KF what if users went to
> the metadata UI and saw data gathered from all their
> applications (Katib, Pipelines, TFJob, notebooks/kale)?
>
> Deployment and application management is a big
> problem; is it a kubeflow problem? Or is it an
> industry/kubernetes problem?
>
> Kubernetes has apps-sig
> <
https://github.com/kubernetes/community/blob/master/sig-apps/README.md> which
> is focused on running and deploying applications.
> Does Kubeflow need its own WG for deployment/app
> management? Rather than creating our own WG should we
> direct folks to participate in apps-sig?
>
> J
>
>
> On Tue, Oct 13, 2020 at 1:12 AM Mathew Wicks
> <
mathew...@gmail.com> wrote:
>
> Hey William,
>
> If my proposal were to be accepted by Kubeflow,
> nothing is to prevent Pipelines from being made
> available outside of the full Kubeflow platform,
> (with the caveat that Kubeflow Pipelines would
> need to rebrand itself to not use the word
> Kubeflow, as Kubeflow is the entire platform).
>
> However, a better solution for your use-case would
> be to install the *generic distribution *and only
> select the Pipelines component.
>
> Naturally, the generic distribution would be
> modular, with only some components required, like
> for example, the Central Dashboard (assuming you
> wanted any form of WebUI).
>
> On Tuesday, 13 October 2020 at 6:59:10 pm UTC+11
>
etern...@gmail.com wrote:
>
> Hi Mathew
>
> I have a bit of issue with the following
> statement.
>
> * Every UI exposed by Kubeflow should
> be* only accessible* through the secure
> central dashboard
>
> Mostly because as mentioned previously, there
> are 2 types of users. Those that use only some
> components of kubeflow, and those who view
> kubeflow as a whole.
>
> For e.g., I would want to be able to use kfp
> independently from kubeflow per se - esp when
> I am running a very light weight distribution
> for a small team.
>
> The part I like about kubeflow is that it is
> easy for me to integrate third party (or
> in-house) components into the kubeflow
> "framework".
>
> Although I would hope central dashboard can be
> modified to be more composible so that I can
> customize w/o maintaining a fork.
>
>
> On Tue, 13 Oct 2020, 8:21 am Mathew Wicks,
> <
mathew...@gmail.com> wrote:
>
> Hi All,
>
> I strongly believe if Kubeflow wants to be
> viewed as the industry standard Kubernetes
> Data platform we must have
> something *beyond* *the name *which ties
> the components together.
> *
> *
> *I view this 'glue' as being three main
> things:*
> * *1.* *A common user-story
> 2. A common distribution
> 3. A common UI (+ single sign-on)
>
> *------------------------------*
> *1 - A common user-story*
> *------------------------------*
> *Clarification:*
>
> * There should be a set
> of *_clear_*, *_well-defined_*,
> and *_opinionated_* user-stories for
> Kubeflow
> * Each *core* *application* should be
> able to justify its contribution to at
> least one of the user-stories
>
> *
> *
> *------------------------------*
> *2 - A common distribution*
> *------------------------------*
> *Clarification:*
>
> * The Kubeflow Project should support
> a *single official distribution *(for
> existing Kuberntes clusters only)
> * To preserve the idea of *Kubeflow as a
> platform*, components of Kubeflow
> should *not* brand themselves as
> Kubeflow (even if they can be
> installed independently)
>
> *Current issues:*
>
> * We are effectively unable to release
> new versions (especially patch
> releases) of Kubeflow, because there
> are too many dependant distributions
> which are NOT controlled by the
> project itself
> * We are distracted by our attempts to
> provide Kubernetes Clusters +
> Kubeflow, this has led to us
> neglecting Kubeflow development, and
> fragmenting into multiple
> forks/distributions
> * Users are confused when they see
> things like Kubeflow Pipelines, and
> assume that is the entire Kubeflow
> platform
>
> *Pending Proposals:*
>
> * Generic Distribution:
>
https://github.com/kubeflow/manifests/issues/1554
> <
https://github.com/kubeflow/manifests/issues/1554>
>
> *
> *
> *------------------------------*
> *3 - A common **UI*
> *------------------------------*
> *Clarification:*
>
> * Each major user-facing component of
> Kubeflow should have *at least some
> form *of UI element
> * Every UI exposed by Kubeflow should
> be*only accessible* through the secure
> central dashboard
>
> *Current issues:*
>
> * Normal users are unaware of all the
> components/features of Kubeflow which
> are not present in the UI. (For
> example, Profiles)
> * Some UI components are not scoped to
> >
+1-917-512-3660 <tel:(917)%20512-3660>
> <
https://groups.google.com/d/msgid/kubeflow-discuss/CAFrZaExxs%3DrtiNNvBcGRHv3b3TPiYYb-KJvvMoAPvGYPmQ4cQg%40mail.gmail.com?utm_medium=email&utm_source=footer
> <
https://groups.google.com/d/msgid/kubeflow-discuss/8478ee5e-592a-42bc-9968-18b469c45b7fn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are
> subscribed to the Google Groups "kubeflow-discuss"
> group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
>
kubeflow-discu...@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com
> <
https://groups.google.com/d/msgid/kubeflow-discuss/f41a5877-8175-442c-abb7-ffc796915871n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed
> to the Google Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
>
kubeflow-discu...@googlegroups.com.
>
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com
> <
https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiZBAp%2BVgm0KY-EZ46UEZybE6-HRcsFpDubWxNuarJ4U%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Luke Marsden | Founder & CEO, MLOps Consulting
> US
+1 415-231-8523 <tel:(415)%20231-8523>
> UK
+44 7791750420 <tel:+44%207791%20750420>
>
https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com
> <
https://groups.google.com/d/msgid/kubeflow-discuss/0a676b58-177b-4489-bd96-ba45a4e932afn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to
kubeflow-discu...@googlegroups.com
>
https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com
> <
https://groups.google.com/d/msgid/kubeflow-discuss/CAHD%3DcxYZW-0jn3M6eRnH2En5ugGmeZyJ9ik0FJTYrOD21aKQqw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "kubeflow-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to
kubeflow-discu...@googlegroups.com
>
https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com
> <
https://groups.google.com/d/msgid/kubeflow-discuss/CACijwiaCz05o%3DS%3DCKMJbE2nArVmurpkvOrxGq%3Dp5oZ2Uzx3c5A%40mail.gmail.com?utm_medium=email&utm_source=footer>.