Jira (PDB-4875) Add option to log PDBQuery correlation uuids in Postgres

2 views
Skip to first unread message

Zachary Kent (Jira)

unread,
Sep 1, 2020, 5:12:02 PM9/1/20
to puppe...@googlegroups.com
Zachary Kent created an issue
 
PuppetDB / New Feature PDB-4875
Add option to log PDBQuery correlation uuids in Postgres
Issue Type: New Feature New Feature
Assignee: Unassigned
Created: 2020/09/01 2:11 PM
Priority: Normal Normal
Reporter: Zachary Kent

In PDB-4834 we added a log-queries config option which logs the AST & SQL produced as a result of queries PDB receives. It may be useful to have a way to correlate these logs with those in Postgres. One way of accomplishing this would be to have Postgres log the PDB uuid used for a query. Then using the Postgres log line prefix created by %c we could more easily correlate any logging on the Postgres side with those generated in PDB.

Read here for more background on log_line_prefix in postgres.

In the PDB code we would need to run `select pdb-uuid;` or a similar statement using the same connection as the SQL query to trigger postgres to log the PDB uuid with the %c id Postgres will add as a log line prefix. This would need to happen in query_eng.clj/stream-query-result and in query_eng.clj/produce-streaming-body. Notice in the 2nd link how we would need to do this in the inner `with-transacted-connection` form. This is due to PDB currently using two hikari connections per query as explained in PDB-4871. If this ticket is resolved you won't need to worry about the nested transactions using different connections.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Zachary Kent (Jira)

unread,
Feb 8, 2021, 7:43:03 PM2/8/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
- In PDB-4834 we added a log-queries config option which logs the AST & SQL produced as a result of queries PDB receives. It may be useful to have a way to correlate these logs with those in Postgres. One way of accomplishing this would be to have Postgres log the PDB uuid used for a query. Then using the Postgres log line prefix created by %c we could more easily correlate any logging on the Postgres side with those generated in PDB. -

- [Read here|https://www.postgresql.org/docs/11/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-WHAT] for more background on log_line_prefix in postgres. -

- In the PDB code we would need to run `select pdb-uuid;` or a similar statement using the same connection as the SQL query to trigger postgres to log the PDB uuid with the %c id Postgres will add as a log line prefix. This would need to happen in [query_eng.clj/stream-query-result|https://github.com/puppetlabs/puppetdb/blob/master/src/puppetlabs/puppetdb/query_eng.clj#L210] and in [query_eng.clj/produce-streaming-body|https://github.com/puppetlabs/puppetdb/blob/master/src/puppetlabs/puppetdb/query_eng.clj#L280]. Notice in the 2nd link how we would need to do this in the inner `with-transacted-connection` form. This is due to PDB currently using two hikari connections per query as explained in PDB-4871. If this ticket is resolved you won't need to worry about the nested transactions using different connections. -

Zachary Kent (Jira)

unread,
Feb 8, 2021, 7:56:03 PM2/8/21
to puppe...@googlegroups.com

David McTavish (Jira)

unread,
Jan 12, 2022, 4:50:01 PM1/12/22
to puppe...@googlegroups.com
David McTavish commented on New Feature PDB-4875
 
Re: Add option to log PDBQuery correlation uuids in Postgres

rob.browning is this a dupe of PDB-4849 ? if so, can you close please

 

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

David McTavish (Jira)

unread,
Jan 12, 2022, 4:50:01 PM1/12/22
to puppe...@googlegroups.com

David McTavish (Jira)

unread,
Feb 10, 2022, 10:26:02 AM2/10/22
to puppe...@googlegroups.com

Alvin Rodis (Jira)

unread,
May 13, 2022, 11:15:01 AM5/13/22
to puppe...@googlegroups.com
Alvin Rodis updated an issue
Change By: Alvin Rodis
Zendesk Ticket Count: 1
Zendesk Ticket IDs: 48171

Alvin Rodis (Jira)

unread,
May 13, 2022, 11:15:03 AM5/13/22
to puppe...@googlegroups.com

Claudia Petty (Jira)

unread,
Jun 21, 2023, 10:56:09 AM6/21/23
to puppe...@googlegroups.com
Claudia Petty updated an issue
Change By: Claudia Petty
Labels: jira_escalated new-feature
This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages