Jira (PDB-4172) Allow metrics during startup

1 view
Skip to first unread message

Dylan Ratcliffe (JIRA)

unread,
Oct 24, 2018, 9:21:02 AM10/24/18
to puppe...@googlegroups.com
Dylan Ratcliffe created an issue
 
PuppetDB / Improvement PDB-4172
Allow metrics during startup
Issue Type: Improvement Improvement
Assignee: Unassigned
Created: 2018/10/24 6:20 AM
Priority: Normal Normal
Reporter: Dylan Ratcliffe

Large HA PuppetDB may take up to an hour (maybe longer) to start due to the initial sync. During this time the HTTP metrics are not available and it is difficult to determine what the sync is doing and if it is progressing as expected. These are metrics that I use personally and are probably some of the more useful are (Bear in mind this is what infludb is calling them, won't map exactly):

  • global_processed (To work out commands per second)
  • global_processing-time
  • queue_depth
  • jvm-metrics_heap-memory_committed
  • jvm-metrics_heap-memory_used
  • jvm-metrics_cpu-usage
  • global_message-persistence-time
  • PDBReadPool_pool_Usage
  • PDBReadPool_pool_Wait
  • PDBWritePool_pool_Usage
  • PDBWritePool_pool_Wait

Having these working as soon as the process starts instead of once it gets to a "running" state would be great, ideally none of the metrics would depend on the fact that PuppetDB is "running" as opposed to "starting"

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Zachary Kent (JIRA)

unread,
Nov 25, 2019, 1:50:04 PM11/25/19
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Team: HA

Nick Burgan (Jira)

unread,
Mar 23, 2020, 2:29:03 PM3/23/20
to puppe...@googlegroups.com
Nick Burgan commented on Improvement PDB-4172
 
Re: Allow metrics during startup

May not be as urgent now that initial sync doesn't take long with streaming provision. However, if we still want metrics during potentially long migrations (e.g. report table migration), we might still want to look into this. Dylan Ratcliffe What do you think?

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Dylan Ratcliffe (Jira)

unread,
Mar 24, 2020, 5:16:03 AM3/24/20
to puppe...@googlegroups.com

I've not seen the impact of streaming provision so I can't really tell. The main issue was that PuppetDB on the secondary would sit there for hours with no metrics at all ad so you would have no way of really knowing what it was doing, how deep the queue was, how far it was from completing etc. If streaming sync (I'm not sure what that is) means that a 2 hour sync would be taken down to a few minutes that would be fine. If customers are still waiting 30min or so though it's good to have metrics.

Nick Burgan (Jira)

unread,
Mar 30, 2020, 2:06:03 PM3/30/20
to puppe...@googlegroups.com
Nick Burgan updated an issue
Change By: Nick Burgan
Story Points: 5

Nick Burgan (Jira)

unread,
Mar 30, 2020, 2:06:03 PM3/30/20
to puppe...@googlegroups.com
Nick Burgan updated an issue
Change By: Nick Burgan
Sprint: HA Team (Ready For Work)

Zachary Kent (Jira)

unread,
Jan 11, 2021, 8:20:04 PM1/11/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Labels: tsr-pdb-backlog

Zachary Kent (Jira)

unread,
Jan 12, 2021, 2:14:03 PM1/12/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Team: HA Ghost

Bogdan Irimie (Jira)

unread,
Jan 13, 2021, 9:43:04 AM1/13/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Story Points: 5 8

Bogdan Irimie (Jira)

unread,
Jan 13, 2021, 9:57:04 AM1/13/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: HA Team (Ready For Work) ghost-27.01.2021

Bogdan Irimie (Jira)

unread,
Jan 27, 2021, 9:03:04 AM1/27/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ghost-27.01.2021 , ready for triage 3

Oana Tanasoiu (Jira)

unread,
Feb 2, 2021, 3:39:03 AM2/2/21
to puppe...@googlegroups.com
Oana Tanasoiu assigned an issue to Oana Tanasoiu
Change By: Oana Tanasoiu
Assignee: Oana Tanasoiu

Bogdan Irimie (Jira)

unread,
Feb 10, 2021, 9:07:05 AM2/10/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ghost-27.01.2021, ghost-10.02.2021 , ready for triage 3

Oana Tanasoiu (Jira)

unread,
Feb 11, 2021, 4:47:02 AM2/11/21
to puppe...@googlegroups.com
Oana Tanasoiu commented on Improvement PDB-4172

Hi Dylan Ratcliffe ,

 

I didn't manage to reproduce the problem with the latest code, also Zachary Kent tried to reproduce it and he also didn't manage to.

I added a test to ensure that this functionality will be preserved in the future.

If you'll see this problem again with newer builds, please reopen the ticket.

Austin Blatt (Jira)

unread,
Feb 22, 2021, 12:16:01 PM2/22/21
to puppe...@googlegroups.com
Austin Blatt updated an issue
Change By: Austin Blatt
Fix Version/s: PDB 6.15.0
Fix Version/s: PDB 7.2.0
Reply all
Reply to author
Forward
0 new messages