Prometheus mysqld_exporter failing with errors.

[Sai Krishna]

Hello everyone, 

Would appreciate any quick assistance in below error. I have upgraded my exporter from 0.12.0 to 0.13.0 but this did not help. Did someone fixed this before? 

6 error(s) occurred:
* [from Gatherer #2] collected metric "mysql_global_variables_validate_password_check_user_name" { gauge:<value:1 > } was collected before with the same name and label values

- Sai

[Brian Candler]

It says that your scrape job is seeing the same metric multiple times with the same labels.

Scrape your exporter directly with curl, and I'm guessing you'll see

mysql_global_variables_validate_password_check_user_name 1

...

mysql_global_variables_validate_password_check_user_name 1

or

mysql_global_variables_validate_password_check_user_name{foo="abc"} 1

...

mysql_global_variables_validate_password_check_user_name{foo="abc"} 1

Either that, or your scrape job is scraping the same exporter multiple times.

[Matthias Rampke]

This may be caused by the mysqld exporter invalidly conflating multiple variables.

Can you please share what your MySQL version is, and what you get from

SHOW GLOBAL VARIABLES LIKE 'validate_password%';

From a quick read, it seems that there is both a plugin, using configuration variables like validate_password_check_user_name, and (as of MySQL 8), a component, using variables like validate_password.check_user_name. The exporter might normalize both to the same metric name, causing the issue.

If that is the case, the SHOW GLOBAL VARIABLES will reveal it. Remove the duplicate (plugin) configuration as per the documentation [1]

/MR

[1] https://dev.mysql.com/doc/refman/8.0/en/validate-password-transitioning.html

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/dc4c8d69-bc6e-4a3f-95bc-0b0d950048bcn%40googlegroups.com.

[Sai Krishna]

Thanks Matt and Brain,

I'm seeing this in MySQL 8.0.21 and 8.0.25. Out of 100 servers, only 4 servers giving this error. 

An error has occurred while serving metrics: 6 error(s) occurred: * [from Gatherer #2] collected metric "mysql_global_variables_validate_password_check_user_name" { gauge:<value:1 > } was collected before with the same name and label values * [from Gatherer #2] collected metric "mysql_global_variables_validate_password_length" { gauge:<value:8 > } was collected before with the same name and label values * [from Gatherer #2] collected metric "mysql_global_variables_validate_password_mixed_case_count" { gauge:<value:1 > } was collected before with the same name and label values * [from Gatherer #2] collected metric "mysql_global_variables_validate_password_number_count" { gauge:<value:1 > } was collected before with the same name and label values * [from Gatherer #2] collected metric "mysql_global_variables_validate_password_special_char_count" { gauge:<value:1 > } was collected before with the same name and label values * [from Gatherer #2] collected metric "mysql_global_status_validate_password_dictionary_file_words_count" { untyped:<value:0 > } was collected before with the same name and label values

[Sai Krishna]

SHOW GLOBAL VARIABLES LIKE 'validate_password%';

+--------------------------------------+--------+

| Variable_name                        | Value  |

+--------------------------------------+--------+

| validate_password.check_user_name    | ON     |

| validate_password.dictionary_file    |        |

| validate_password.length             | 8      |

| validate_password.mixed_case_count   | 1      |

| validate_password.number_count       | 1      |

| validate_password.policy             | MEDIUM |

| validate_password.special_char_count | 1      |

| validate_password_check_user_name    | ON     |

| validate_password_dictionary_file    |        |

| validate_password_length             | 8      |

| validate_password_mixed_case_count   | 1      |

| validate_password_number_count       | 1      |

| validate_password_policy             | MEDIUM |

| validate_password_special_char_count | 1      |

+--------------------------------------+--------+

[Sai Krishna]

Its weird that all other exporters with same versions working fine. 

[Brian Candler]

You can see you have both "validate_password.check_user_name" and "validate_password_check_user_name".  That's the problem.

Metric names cannot have dots, so "validate_password.check_user_name"  is exposed as "validate_password_check_user_name", and duplicates the real "validate_password_check_user_name"

[Sai Krishna]

Is it possible to ignore from Prometheus exporter config.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/50a86f38-6af9-4dc3-b38e-56197656e439n%40googlegroups.com.