Greenplum memory consumption

[muhammad ali]

Hello Experts,

Current GP specs:
6 node cluster(1 master, 1 standby master, 4 worker nodes)

Each worker node has 1250GB memory

Greenplum conf parameters:

1: gp_resource_group_memory_limit = 0.85%
2: statement_mem=2560MB
3: gp_vmem_protect_limit=103306
4: gp_vmem_protect_segworker_cache_limit=500
5: gp_workfile_limit_per_query=1073741824
6: gp_resource_manager='group'
7: gp_resgroup_memory_policy=eager_free
8: gp_cached_segworkers_threshold=15

OS Memory parameters:

1: vm.overcommit_ratio = 2
2: vm.overcommit_memory = 90
3: vm.swapiness = 1
kernel.shmem and other parameters are already set.

I've also attached the output of free -g command. Now around 1137GB memory is in buff/cache. When we check from GPCC, memory utilization is around 2-3% in total.

1: I want to ask how memory is utilized in Greenplum cluster? Does it works with OS cache in backend?

2: Are there any other configuration parameters(OS/GP) which can enhance GP cluster performance if we have the available memory resources.

3: would there be any bottlenecks if we reduce the memory as per our maximum utilization which is not greater than 5%.

Thanks and Regards,
Muhammad Ali

[Luis Filipe de Macedo]

Hey Ali,

 

How many segments do you have on each node? How many cores do you have on each node?

 

Usually Greenplum is really good at memory usage. It Only really uses memory on queries that aggregate data and very large joins.

 

 

Rgds,

 

Luis Filipe R Macedo

Adv Data Engineering

VMware Data Solutions

+55 11 988608596

Take care of the customers and the rest takes care of itself

--

 

From: muhammad ali <engmuha...@gmail.com>
Date: Friday, 17 March 2023 19:54
To: Greenplum Users <gpdb-...@greenplum.org>
Subject: [gpdb-users] Greenplum memory consumption

!! External Email

--
You received this message because you are subscribed to the Google Groups "Greenplum Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gpdb-users+...@greenplum.org.
To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/87264676-b66b-4197-b634-4c18479b30c2n%40greenplum.org.

 

!! External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.