Jira (PDB-5068) Add sync e2e test which isolates the in-mem initial sync path

19 views
Skip to first unread message

Zachary Kent (Jira)

unread,
Mar 18, 2021, 4:02:03 PM3/18/21
to puppe...@googlegroups.com
Zachary Kent created an issue
 
PuppetDB / Improvement PDB-5068
Add sync e2e test which isolates the in-mem initial sync path
Issue Type: Improvement Improvement
Assignee: Unassigned
Created: 2021/03/18 1:01 PM
Priority: Normal Normal
Reporter: Zachary Kent

We recently saw a minor bug get past the sync e2e tests in pdbext because the initial sync path enqueues commands in stockpile when the initial command submission errors. This caused errors in the command ingestion of initial sync to be ignored because the commands were stored in PDB via the normal command ingestion path.

We should add a test which checks that the initial sync path is working correctly without needing to enqueue commands after a failure.

The process-or-enqueue-command func is what controls command submission for the initial sync and blocking sync paths. We might be able to redef the make-durable-enqueue-command func to throw an error if called and then arrange a test which needs to transfer records during initial sync. Something along these lines should give us a test which will pop if we break the initial sync path.

The initial-sync test can be used as an example of how to arrange the testing macros so there is data which needs to be transferred during an initial sync.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Zachary Kent (Jira)

unread,
Mar 18, 2021, 4:03:03 PM3/18/21
to puppe...@googlegroups.com

Zachary Kent (Jira)

unread,
Mar 18, 2021, 4:05:53 PM3/18/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Acceptance Criteria: * A sync e2e test which fails if the initial sync path is broken

Bogdan Irimie (Jira)

unread,
Mar 30, 2021, 3:49:03 AM3/30/21
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
May 19, 2021, 4:03:01 AM5/19/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Story Points: 3
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Bogdan Irimie (Jira)

unread,
May 19, 2021, 4:03:04 AM5/19/21
to puppe...@googlegroups.com

Andrei Filipovici (Jira)

unread,
May 20, 2021, 10:34:04 AM5/20/21
to puppe...@googlegroups.com
Andrei Filipovici commented on Improvement PDB-5068
 
Re: Add sync e2e test which isolates the in-mem initial sync path

The time shift export utility accepts 3 parameters:

  • [Mandatory] -i / --input
    Path to the .tgz pdb export, which will be shifted.
  • [Optional] -o / --output
    Path to the where the shifted export will be saved. If no path is given, the archive will be stored in the same place as the input export, but the export name will be prefixed with "shifted". (e.g input-export.tgz will become shifted-input-export.tgz). If the output path doesn't end with .tgz it will be added.
  • [Optional]-t / --shift-to-time
    Timestamp to which all the export timestamp will be shifted. If it's not provided, the system's current timestamp will be used.

The time shifting tool will convert all timestamps to a Zero timezone format. e.g timestamp like: 2015-03-26T10:58:51+10:00 will become 2015-03-26T11:58:51Z

Andrei Filipovici (Jira)

unread,
May 20, 2021, 10:42:02 AM5/20/21
to puppe...@googlegroups.com
Andrei Filipovici updated an issue
 
Change By: Andrei Filipovici
Comment:
The time shift export utility accepts 3 parameters:
- [Mandatory] -i / --input
Path to the .tgz pdb export, which will be shifted.

- [Optional] -o / --output
Path to the where the shifted export will be saved. If no path is given, the archive will be stored in the same place as the input export, but the export name will be prefixed with "shifted". (e.g input-export.tgz will become shifted-input-export.tgz). If the output path doesn't end with .tgz it will be added.

- [Optional]-t / --shift-to-time
Timestamp to which all the export timestamp will be shifted. If it's not provided, the system's current timestamp will be used.

The time shifting tool will convert all timestamps to a Zero timezone format. e.g timestamp like: 2015-03-26T10:58:51+10:00 will become 2015-03-26T11:58:51Z

Andrei Filipovici (Jira)

unread,
May 21, 2021, 3:07:04 AM5/21/21
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Jun 2, 2021, 3:30:02 AM6/2/21
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ghost-2.06.2021 , ready for triage 3

Austin Blatt (Jira)

unread,
Jun 17, 2021, 5:09:02 PM6/17/21
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages