gpcc_queries_history no longer logging after settings change

[Matias Montroull]

Hi,

I've changed these parameters:

gpconfig -c log_rotation_size -v 0 --skipvalidation
gpconfig -c log_truncate_on_rotation -v on --skipvalidation
gpconfig -c log_filename -v \'gpdb-%d.csv\' --skipvalidation

Before, this log_truncate_on_rotation was off and the log_filename had a different string.

The aim behind this was to generate 30 days rotating logs.

Then restarted GP and now the gpcc_queries_history is no longer collecting information, although I can see log files under /data/master/gpseg-1/pg_log on master and segment hosts (under /data1/primary/gpsegX/pg_log)

Did I miss anything?

Thanks

[Danilo Fortunato]

Matias,

I don't know the reason why the queries_history is no longer collecting information.

However, I suggest you to read this article of the Greenplum knowledge base, since some of your commands are in contrast with the informations given in the article.

How to Manage Log Files

https://community.pivotal.io/s/article/How-to-Manage-Log-Files?language=en_US

Regards,

Danilo Fortunato

[Matias Montroull]

Thanks, I restarted GP DB and everything started to work normally

[Matias Montroull]

Hi, 

This issue came back again, I looked at the document shared by Danilo but didn't see anything wrong on my server, to be honest not sure why is no longer collecting the logs in the table, is there a service that runs in the server that transfers the CSV data into the table?

Thks

[Danilo Fortunato]

Hello Matias,

queries_history is one of the table of gpperfmon service database.

The service that loads this table and the other tables of gpperfmon is described in this paragraph:

https://docs.vmware.com/en/VMware-Tanzu-Greenplum/6/greenplum-database/GUID-ref_guide-gpperfmon-dbref.html?hWord=N4IghgNiBcII4FcCmAnAlkgzgfQBZswBcB7FATxAF8g

I would check if the process mentioned in this paragraph does exist on your system, as well as the other files mentioned.

Regards,

Danilo Fortunato

---

Da: Matias Montroull <mati...@gmail.com> (mati...@gmail.com)
Inviato: Venerdì 13 Gennaio 2023 13:40
A: Greenplum Users (gpdb-...@greenplum.org)
Cc: Matias Montroull, Danilo Fo..., Matias Montroull (mati...@gmail.com, danilo.fo...@gmail.com, gpdb-...@greenplum.org)
Oggetto: Re: gpcc_queries_history no longer logging after settings change

 

[Matias Montroull]

Hi Danilo, the service gpmon is indeed running on my system, what I can't see is the gpsmon agent/service, but on another system we have where it works, we can't find that service either so not sure it is visible. What I can tell for sure is that gpmon is running.

Thanks

[Matias Montroull]

It is worth mentioning that there was an installation of gpcc version 6.7.2 which was rolled back due to issues.

Maybe I need to uninstall everything related to command center and reinstall, are there instructions on how to clean and start over?

Thanks

[Danilo Fortunato]

Hi Matias,

yes, the Greenplum Command Center documentation describes how to uninstall it:

https://docs.vmware.com/en/VMware-Tanzu-Greenplum-Command-Center/6.7/tanzu-greenplum-command-center/GUID-topics-setup-uninstall.html

Please note that I haven't a direct experience on uninstalling Greenplum Command Center.

Regards,

Danilo Fortunato

---

Da: Matias Montroull <mati...@gmail.com> (mati...@gmail.com)

Inviato: Lunedì 30 Gennaio 2023 15:57
A: Danilo Fortunato (danilo.fo...@gmail.com)
Cc: Greenplum Users (gpdb-...@greenplum.org)

[Kevin Huang]

Hi Matias,

queries_history table is populated by gpmmon/gpsmon process which is controlled by the gp_enable_gpperfmon GUC. This table and GUC is considered legacy and is actually not used/supported with GPCC 6.x.

gpcc_queries_history is a different table that is populated by GPCC 6.x background process called ccagent. As long as GPCC is started and ccagent is running on all master and all segments then the data collection should be happening. 

1.) Make sure Enable GPCC history data collection is turned on: https://docs.vmware.com/en/VMware-Tanzu-Greenplum-Command-Center/6.7/tanzu-greenplum-command-center/GUID-topics-ui-admin-settings.html  
2.) If it is then maybe check the master log against gpcc_queries_history and see if there's an error being reported.

Kevin

[Matias Montroull]

Hi Kevin,

I checked and the GPCC collector is enabled:

Also, the process is running on the server:

The history stops at a date and time when we installed GPCC 6.7.2 (then we rolled back to 6.3.1 as it was making the server fail)

Everything seems to be fine, but for some reason the queries are not loaded in the gpcc_queries_history.

Other gpcc tables are loaded fine, such as gpcc_pg_log_history for example.

Thanks

[Joe Manning]

Hi.

 

Changing the version of GPCC may need to change the metrics_collector extension.

See https://docs.vmware.com/en/VMware-Tanzu-Greenplum-Command-Center/6.7/tanzu-greenplum-command-center/GUID-topics-install.html for more information on the metrics collector extension.

 

Regards,

joe.

 

From: Matias Montroull <mati...@gmail.com>
Date: Tuesday, 31 January 2023 at 13:31
To: Kevin Huang <kchua...@gmail.com>
Cc: Greenplum Users <gpdb-...@greenplum.org>, pvtl-cont-danilo.fortunato.2 <danilo.fo...@gmail.com>
Subject: [gpdb-users] Re: gpcc_queries_history no longer logging after settings change

!! External Email

Hi Kevin,

 

I checked and the GPCC collector is enabled:

 

Also, the process is running on the server:

 

--
You received this message because you are subscribed to the Google Groups "Greenplum Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gpdb-users+...@greenplum.org.
To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/CAKKNfuL7zNt4k0JvYOHwsihUp1eKc%2BU1vOE3qYZJ2f181F3pzg%40mail.gmail.com.

 

!! External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.

 

[Matias Montroull]

Interesting,

The extension doesn't exists in my greenplum installation (shall I create it?), and looking at the ones available I can see this:

I'm using Greenplum V 6.13.0 so I can't see it in there (it goes up to 6.11.1)

[Kevin Huang]

I'm unable to check myself, but usually if the metrics collector for your Greenplum version is not there then you may not need to update it. It usually prompts for this metrics collector update in the installation. You can use this command to double check what version of metrics collector is installed:  gppkg -q --all

I would also grep for gpcc_queries_history in the master log to see if there's any errors reported.

[Matias Montroull]

Thanks, I ran this:

gppkg -q --all

And I can only see madlib and gpbackup/restore installed in there, no entries related to the metrics collector.

I also checked the master log and there's no errors related to gpcc_queries_history..

[Joe Manning]

Hi, Matias.

 

Try installing the appropriate gppkg from the “gppkg” directory in the GPCC installation directory.

Example: /usr/local/greenplum-cc-6.7.0/gppkg

 

Regards,

joe.

 

From: Matias Montroull <mati...@gmail.com>
Date: Tuesday, 31 January 2023 at 19:55
To: Kevin Huang <kchua...@gmail.com>
Cc: Greenplum Users <gpdb-...@greenplum.org>, pvtl-cont-danilo.fortunato.2 <danilo.fo...@gmail.com>, Joe Manning <mann...@vmware.com>
Subject: Re: [gpdb-users] Re: gpcc_queries_history no longer logging after settings change

!! External Email

Thanks, I ran this:

gppkg -q --all

 

And I can only see madlib and gpbackup/restore installed in there, no entries related to the metrics collector.

 

I also checked the master log and there's no errors related to gpcc_queries_history..

 

El mar., 31 de enero de 2023 4:13 p. m., Kevin Huang <kchua...@gmail.com> escribió:

I'm unable to check myself, but usually if the metrics collector for your Greenplum version is not there then you may not need to update it. It usually prompts for this metrics collector update in the installation. You can use this command to double check what version of metrics collector is installed:  gppkg -q --all

 

I would also grep for gpcc_queries_history in the master log to see if there's any errors reported.

On Tuesday, January 31, 2023 at 9:05:52 AM UTC-8 mati...@gmail.com wrote:

Interesting,

 

The extension doesn't exists in my greenplum installation (shall I create it?), and looking at the ones available I can see this:

 

To view this discussion on the web visit https://groups.google.com/a/greenplum.org/d/msgid/gpdb-users/CAKKNfu%2BMUh8LR2uFvLY0Hc9biGmkpv5P85H4bxeNdOsrRoWDVg%40mail.gmail.com.

[Matias Montroull]

Hi Joe,

This is that I don't see 6.13.0 package in there (that's the GPDB version I have)..

Thanks

[Joe Manning]

Hi.

 

You need a newer version of GPCC to get the gppkg for the newer GPDB.

 

What issues did you have with GPCC 6.7.2 when it was installed?

I presume you did not install the gppkg for that version when you installed it as the gppkg is not present at the moment.

[Matias Montroull]

I can try to reinstall 6.7.2

At the time, the database was entering recovery mode over and over. The GPCC installation failed in the standby master (permission denied). We had to rollback quickly so we are not sure if the issue was around permissions on standby master or something else