Greenplum for Kubernetes

[Stuart Hayes]

I'm looking into the feasibility of running Greenplum on Kubernetes, likely openshift in a production environment.

Does anybody have any experience doing this, is there are a commercial Greenplum for k8s?

Thanks

Stuart

[Shine Zhang]

You can find more information from https://network.tanzu.vmware.com/products/greenplum-for-kubernetes/.

It's end-of-life, no longer supported by Greenplum for now.

The open source is at https://github.com/greenplum-db/greenplum-for-kubernetes.

Thanks,

Shine

--
You received this message because you are subscribed to the Google Groups "Greenplum Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gpdb-users+...@greenplum.org.
To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/59061bae-2cbc-48ae-a5ce-0186409cbb3dn%40greenplum.org.

This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.

[Artem Ustsov]

Dear Shine, Stuart.
While searching for answers to the current state of greenplum in k8s, I came across your post.

Please tell me, do you know the reasons for refusing to develop the product in this direction? Have there been any technological difficulties or is it more due to a lack of market interest?
I will be very grateful for any answers. Thank you!

суббота, 27 января 2024 г. в 01:36:11 UTC+3, Shine Zhang:

[Karen Huddleston]

Hello Artem,

I was an engineer on the GP4K product for several years. My understanding was that it was mostly a lack of market interest. We built a decent amount of functionality in the product but were unable to get any customers so the project was discontinued.

- Karen

To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/6aad9d52-51d8-4388-87f0-bcfe783cd0b8n%40greenplum.org.

[Artem Ustsov]

Hi Karen!
Thanks for the revelation! It's surprising, actually, that they stopped supporting this. However, many official links to documentation, articles or videos about deployment have simply been removed

Is it really possible to tackle this issue on your own? As far as I could find information - the latest version of GP4K  2.3.0 is equivalent to 6.11.1, and I want to try to deploy 6.23.5.
What is the main difference between the gp package and gp4k? Is it possible to deploy standard gp in k8s without using gp4k?

Thanks

четверг, 14 марта 2024 г. в 00:05:03 UTC+3, Karen Huddleston:

[Ivan Novick]

Why not try to just deploy on K8S yourself using Greenplum software.  It should be doable without any special codes from Greenplum.  I would just recommend using mirrorless mode.

Ivan

=====================================================

Ivan Novick

Manager of Product Management, Tanzu Data Services

Tanzu by Broadcom

ivan....@broadcom.com

+1-408-230-6491

To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/3df9f94f-743b-4ff5-8baa-86d00a29796bn%40greenplum.org.

[Ivan Novick]

It would be a good idea for folks to try bundling GP in K8S containers of their own.  One of the key aspects is the customization of the image, which end users can totally control if they build their own.  then with mirroless, ideally you just start and baby sit all the processes to be running and it should be working fine.  

=====================================================

Ivan Novick

Manager of Product Management, Tanzu Data Services

Tanzu by Broadcom

ivan....@broadcom.com

+1-408-230-6491

On Thu, Mar 14, 2024 at 10:53 AM Luis Filipe De Macedo <luis-filip...@broadcom.com> wrote:

Also, I don't think we added or removed anything from the core product, It should run fine in k8s as long as you know what you are doing. 

It's just that I doubt the k8s infra will give you good performance and performance is the main reason to use Greenplum.

Rgds,

Luis F R Macedo

Solutions Architect

VMware by Broadcom

Phone: +55 11 98860-8596

Take care of the customers and the rest takes care of itself

To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/CAG0QGr_-LPOu_0PxnDAmQB-mH43YDzj2-oxzn63VLK4EsZrJ9g%40mail.gmail.com.

[Karen Huddleston]

Yeah I would agree. The previous greenplum for kubernetes build wouldn't work since the version of Greenplum is old and it supports an old version of Kubernetes, but I think you could manually deploy greenplum in kubernetes if you wanted.

[Stuart Hayes]

Thanks for all the responses, so a couple of questions:

If we use enterprise (ie licensed) Greenplum running in k8s with a custom OS image, will this be supported?

Also why mirror less?

Thanks

Stuart

[Karen Huddleston]

I would note that the actual Greenplum for Kubernetes implementation had some extra work done to restart the psql process on the segments when a segment went down. Kubernetes will bring the pod back quickly, but vanilla Greenplum will not automatically recover. So actually it may be better to run in a mirrored configuration if you are just using Greenplum out of the box. Otherwise if a segment crashes it will not automatically recover the postgres process. Our Greenplum Virtual implementation also has a little extra work to support the mirrorless auto recovery but that isn't yet available for a vanilla Greenplum deployment.

- Karen

On Tue, Mar 19, 2024 at 7:01 AM Luis Filipe De Macedo <luis-filip...@broadcom.com> wrote:

Staurt,

As long as you are using the provided binaries and you have selected a supported OS version for the image you should be fine.

Mirrorless assumes that the storage below has its own HA and that dead servers will be restarted automatically which is the case for a k8s implementation. Also you will save up on storage.

[Ivan Novick]

Can we use the Greenplum virtual service, Karen?  that is being including in the product i think, to handle restarts automatically?

Ivan

[Karen Huddleston]

I don't believe we've tested running those tools outside of a greenplum virtual environment with all of the surrounding tooling. There would likely be work required in separating out the postmaster service. For now I think it's probably best to run with a mirrored configuration until we have supported tooling available.