Jira (PDB-5105) Benchmark console query performance and identify slow queries

[Bogdan Irimie (Jira)]

Bogdan Irimie created an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries Issue Type: Task Assignee: Unassigned Created: 2021/04/14 7:57 AM Priority: Normal Reporter: Bogdan Irimie Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Labels: 2021-o1-kr2 c3

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Labels: 2021-o1-kr2 c3 c4

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Labels: 2021-o1-kr2 c1 c3 c4

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Priority: Normal High

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries Change By: Bogdan Irimie

We should benchmark the queries gathered in PDB-5043 with a PDB instance that has data for 10, 100, 1000, 10000 and 100000 nodes. PDB can be field with synthetic data generated by the benchmark tool from PDB.

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Acceptance Criteria: A CSV with timing for all queries from PE console that ran against a PDB with data for 10, 100, 1000, 10000, 100000 nodes.

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Labels: 2021-o1-kr2 c3 c4

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Labels: 2021-o1-kr2 c1 c3 c4

Add Comment

[Zachary Kent (Jira)]

Zachary Kent commented on PDB-5105

Re: Benchmark console query performance and identify slow queries Related to this ticket CD4PE has gathered some of the common PDB queries they use in a Google doc linked to CDPE-4097. That should be another source of possible queries we could help improve.

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105

Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Story Points: 2

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Sprint: HAHA/Grooming 2

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Sprint: Ghost-5.05.2021

Add Comment

[Eric Thompson (Jira)]

Eric Thompson updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Eric Thompson Epic Link: PE-31891

Add Comment

[Eric Thompson (Jira)]

Eric Thompson updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Eric Thompson Priority: High Critical

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie commented on PDB-5105

Re: Benchmark console query performance and identify slow queries We should use n1/n2 or vmpooler

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105

Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Sprint: ghost-30.06.2021

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Acceptance Criteria: A One or multiple CSV with timing for all queries from PE console that ran against a PDB with data for 10, 100, 1000, 10000, 100000 nodes.

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici assigned an issue to Andrei Filipovici

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Andrei Filipovici Assignee: Andrei Filipovici

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici commented on PDB-5105

Re: Benchmark console query performance and identify slow queries To run the tests on vmPooler, you need to install PE, python3 and python3-devel. Then copy over the load-test folder from the project and run locust

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici updated an issue

PuppetDB / PDB-5105

Benchmark console query performance and identify slow queries

Change By: Andrei Filipovici

We should benchmark the queries gathered in PDB-5043 with a PDB instance that has data for 10, 100, 1000, 10000 and 100000 nodes. PDB can be field with synthetic data generated by the benchmark tool from PDB.

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Bogdan Irimie Sprint: ghost-30.06.2021 , ready for triage 3

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Andrei Filipovici Attachment: Screenshot 2021-06-30 at 16.53.07.png

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici commented on PDB-5105

Re: Benchmark console query performance and identify slow queries I added the 3 sandboxes (1000, 10.000, 100.000) to pdb-n1-prod.ops.puppetlabs.net, at: /home/andrei.filipovici/sandboxes I added in the database section of pdb.ini, in all sandboxes the following settings: node-ttl = 0s node-purge-ttl = 0s report-ttl = 0s resource-events-ttl = 0s Ingestion time for the 10.000 nodes sandbox was a little over half an hour and for the 100.000 nodes was roughly 8 and a half hours. Maximum JVM heap size for the 100.000 nodes sandbox was 10Gb. In the attachment is a print screen for the 100.000 nodes sandbox after it finished ingestion.

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici updated an issue

PuppetDB / PDB-5105

Benchmark console query performance and identify slow queries

Change By: Andrei Filipovici Attachment: Screenshot 2021-06-30 at 16.53.07.png

Add Comment

[Andrei Filipovici (Jira)]

Andrei Filipovici updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Andrei Filipovici Attachment: Screenshot 2021-07-01 at 09.09.04.png Attachment: Screenshot 2021-06-30 at 21.50.12.png Attachment: Screenshot 2021-06-30 at 21.46.48.png Attachment: Screenshot 2021-06-30 at 21.46.15.png

Add Comment

[Rob Browning (Jira)]

Rob Browning updated an issue

PuppetDB / PDB-5105 Benchmark console query performance and identify slow queries

Change By: Rob Browning Fix Version/s: PDB 6.18.0 Fix Version/s: PDB 7.5.0

Add Comment