Jira (PDB-3518) idx_certnames_latest_report_id causes timeouts for filter in console overview

4 views
Skip to first unread message

Thomas Kishel (JIRA)

unread,
May 17, 2017, 4:15:02 PM5/17/17
to puppe...@googlegroups.com
Thomas Kishel updated an issue
 
PuppetDB / Bug PDB-3518
idx_certnames_latest_report_id causes timeouts for filter in console overview
Change By: Thomas Kishel
Summary: idx_certnames_latest_report_id  slows  causes timeouts for  filter  performance  in console overview
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Thomas Kishel (JIRA)

unread,
May 17, 2017, 4:17:02 PM5/17/17
to puppe...@googlegroups.com
Thomas Kishel updated an issue
Potential regression ...

User with the following amount of data:

{code}
public | reports | table | pe-puppetdb | 42 GB | 
public | reports_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | resource_events | table | pe-puppetdb | 3736 MB | 
public | resource_params | table | pe-puppetdb | 31 MB |
{code}

{code}
pe-puppetdb=# select count(*) from resource_events ;
count 
---------- 
10039459 
(1 row)
{code}

Adding the index from PDB-3053:

{code}CREATE UNIQUE INDEX CONCURRENTLY idx_certnames_latest_report_id ON certnames(latest_report_id){code}

... resolved slow/ Resolved timeout issues in the Console Overview, but created  slow/ timeout issues when using the "Unresponsive for N+ hours" Filter in the Console Overview.

I will post queries {code}
2017-05-15 10:11:03.131 EDT [db:pe-puppetdb
,  and ask user for EXPLAIN ANALYZE output sess :  with and without the index 5919ab15 . 2fd7,pid:12247,vtid:59/1655914,tid:0] LOG:  duration: 129949.316 ms  execute S_1012/C_1013: 
{code}

{code}
SELECT certnames.certname AS certname, reports_environment.environment AS report_environment, reports.end_time AS report_timestamp, encode(reports.hash::bytea, 'hex') AS latest_report_hash, report_statuses.status AS latest_report_status, reports.noop AS latest_report_noop, reports.noop_pending AS latest_report_noop_pending, reports.cached_catalog_status AS cached_catalog_status, reports.corrective_change AS latest_report_corrective_change FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE ((reports.end_time < $1) AND ((certnames.certname) in  ( (SELECT certnames.certname AS certname FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE (certnames.deactivated IS NULL AND certnames.expired IS NULL)) ) )) ORDER BY reports.end_time DESC, reports.end_time DESC LIMIT 50 OFFSET 0;
{code}

Thomas Kishel (JIRA)

unread,
May 17, 2017, 5:11:02 PM5/17/17
to puppe...@googlegroups.com
Thomas Kishel updated an issue
Change By: Thomas Kishel
Attachment: explain_withindex_slow.txt
Attachment: explain_withoutindex_fast.txt

Thomas Kishel (JIRA)

unread,
May 17, 2017, 5:11:03 PM5/17/17
to puppe...@googlegroups.com
Thomas Kishel updated an issue
Potential regression ...

User with the following amount of data:

{code}
pe-puppetdb=# \d+ 
List of relations 
Schema | Name | Type | Owner | Size | Description 
--------+----------------------------+----------+-------------+------------+------------- 
public |  catalog_resources | table | pe-puppetdb | 629 MB | 
public | catalogs | table | pe-puppetdb | 7728 kB | 
public | catalogs_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | catalogs_transform_id_seq1 | sequence | pe-puppetdb | 8192 bytes | 
public | certname_facts | view | pe-postgres | 0 bytes | 
public | certname_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | certnames | table | pe-puppetdb | 1880 kB | 
public | edges | table | pe-puppetdb | 279 MB | 
public | environments | table | pe-puppetdb | 16 kB | 
public | environments_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | fact_paths | table | pe-puppetdb | 5264 kB | 
public | fact_paths_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | fact_values | table | pe-puppetdb | 97 MB | 
public | fact_values_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | facts | table | pe-puppetdb | 182 MB | 
public | factsets | table | pe-puppetdb | 2712 kB | 
public | factsets_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | producers | table | pe-puppetdb | 16 kB | 
public | producers_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | report_statuses | table | pe-puppetdb | 16 kB | 
public | report_statuses_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public |
 reports | table | pe-puppetdb | 42 GB | 

public | reports_id_seq | sequence | pe-puppetdb | 8192 bytes | 
public | resource_events | table | pe-puppetdb |  3736  3563  MB | 
public | resource_params | table | pe-puppetdb |
 31  38  MB |  
public | resource_params_cache | table | pe-puppetdb | 23 MB | 
public | schema_migrations | table | pe-puppetdb | 8192 bytes | 
public | value_types | table | pe-puppetdb | 8192 bytes | 
(28 rows)

{code}
pe-puppetdb=# select count(*) from certnames; 
count 
------- 
5236 
(1 row)

{code}
pe-puppetdb=# select count(*) from resource_events ;

 
count 
---------
 
1217849 
(1 row)

pe
- puppetdb=# select count(*) from reports;  
10039459 count  
--------- 
1079718 
(1 row)
{code}

Adding the index from PDB-3053:

{code}CREATE UNIQUE INDEX CONCURRENTLY idx_certnames_latest_report_id ON certnames(latest_report_id){code}

Resolved timeout issues in the Console Overview, but created timeout issues when using the "Unresponsive for N+ hours" Filter in the Console Overview.

{code}
2017-05-15 10:11:03.131 EDT [db:pe-puppetdb,sess:5919ab15.2fd7,pid:12247,vtid:59/1655914,tid:0] LOG:  duration: 129949.316 ms  execute S_1012/C_1013: 

{code}

{code}
SELECT certnames.certname AS certname, reports_environment.environment AS report_environment, reports.end_time AS report_timestamp, encode(reports.hash::bytea, 'hex') AS latest_report_hash, report_statuses.status AS latest_report_status, reports.noop AS latest_report_noop, reports.noop_pending AS latest_report_noop_pending, reports.cached_catalog_status AS cached_catalog_status, reports.corrective_change AS latest_report_corrective_change FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE ((reports.end_time < $1) AND ((certnames.certname) in  ( (SELECT certnames.certname AS certname FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE (certnames.deactivated IS NULL AND certnames.expired IS NULL)) ) )) ORDER BY reports.end_time DESC, reports.end_time DESC LIMIT 50 OFFSET 0;
{code}

Thomas Kishel (JIRA)

unread,
May 17, 2017, 5:12:02 PM5/17/17
to puppe...@googlegroups.com
Thomas Kishel updated an issue
Change By: Thomas Kishel
Comment: A comment with security level 'Developers' was removed.

Thomas Kishel (JIRA)

unread,
May 18, 2017, 2:37:07 PM5/18/17
to puppe...@googlegroups.com
Thomas Kishel commented on Bug PDB-3518
 
Re: idx_certnames_latest_report_id causes timeouts for filter in console overview

Reviewing the explains, a difference is that, with the idx_certnames_latest_report_id index, an initial sort is not performed.

Russell Mull (JIRA)

unread,
May 23, 2017, 12:56:02 PM5/23/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
May 23, 2017, 12:56:03 PM5/23/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
May 23, 2017, 5:15:04 PM5/23/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
May 23, 2017, 5:16:03 PM5/23/17
to puppe...@googlegroups.com

Karen Van der Veer (JIRA)

unread,
May 31, 2017, 5:09:05 PM5/31/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
Jun 5, 2017, 7:26:03 PM6/5/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Fix Version/s: PDB 5.0.0
Fix Version/s: PDB 4.4.1
Fix Version/s: PDB 4.2.3.4

Russell Mull (JIRA)

unread,
Jun 6, 2017, 5:42:03 PM6/6/17
to puppe...@googlegroups.com
Russell Mull updated an issue
public | resource_events | table | pe-puppetdb | 3563 MB | 
public | resource_params | table | pe-puppetdb | 38 MB | 

public | resource_params_cache | table | pe-puppetdb | 23 MB | 
public | schema_migrations | table | pe-puppetdb | 8192 bytes | 
public | value_types | table | pe-puppetdb | 8192 bytes | 
(28 rows)

pe-puppetdb=# select count(*) from certnames; 
count 
------- 
5236 
(1 row)

pe-puppetdb=# select count(*) from resource_events; 
count 
--------- 
1217849 
(1 row)

pe-puppetdb=# select count(*) from reports; 
count 
--------- 
1079718 
(1 row)
{code}

Adding the index from PDB-3053:

{code}CREATE UNIQUE INDEX CONCURRENTLY idx_certnames_latest_report_id ON certnames(latest_report_id){code}

Resolved timeout issues in the Console Overview, but created timeout issues when using the "Unresponsive for N+ hours" Filter in the Console Overview.

{code}
2017-05-15 10:11:03.131 EDT [db:pe-puppetdb,sess:5919ab15.2fd7,pid:12247,vtid:59/1655914,tid:0] LOG:  duration: 129949.316 ms  execute S_1012/C_1013: 
{code}

{code}
SELECT certnames.certname AS certname, reports_environment.environment AS report_environment, reports.end_time AS report_timestamp, encode(reports.hash::bytea, 'hex') AS latest_report_hash, report_statuses.status AS latest_report_status, reports.noop AS latest_report_noop, reports.noop_pending AS latest_report_noop_pending, reports.cached_catalog_status AS cached_catalog_status, reports.corrective_change AS latest_report_corrective_change FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE ((reports.end_time < $1) AND ((certnames.certname) in  ( (SELECT certnames.certname AS certname FROM certnames LEFT JOIN catalogs ON catalogs.certname = certnames.certname LEFT JOIN factsets fs ON certnames.certname = fs.certname LEFT JOIN reports ON certnames.latest_report_id = reports.id LEFT JOIN environments catalog_environment ON catalog_environment.id = catalogs.environment_id LEFT JOIN report_statuses ON reports.status_id = report_statuses.id LEFT JOIN environments facts_environment ON facts_environment.id = fs.environment_id LEFT JOIN environments reports_environment ON reports_environment.id = reports.environment_id WHERE (certnames.deactivated IS NULL AND certnames.expired IS NULL)) ) )) ORDER BY reports.end_time DESC, reports.end_time DESC LIMIT 50 OFFSET 0;
{code}

Russell Mull (JIRA)

unread,
Jun 6, 2017, 6:45:03 PM6/6/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
Jun 6, 2017, 6:46:03 PM6/6/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Release Notes Summary: Improved performance for some queries involving each node's latest report. 

Zachary Kent (Jira)

unread,
Mar 10, 2021, 3:34:04 PM3/10/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Sprint: PuppetDB 2017-05-31 , Release Engineering Hopper
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Morgan Rhodes (Jira)

unread,
Mar 11, 2021, 2:26:02 PM3/11/21
to puppe...@googlegroups.com
Morgan Rhodes updated an issue
Change By: Morgan Rhodes
Sprint: PuppetDB 2017-05-31, Release Engineering Hopper
Reply all
Reply to author
Forward
0 new messages