Jira (PDB-5058) Fix sync

[Bogdan Irimie (Jira)]

Bogdan Irimie created an issue

PuppetDB / PDB-5058 Fix sync Issue Type: Bug Assignee: Unassigned Created: 2021/03/10 6:43 AM Priority: Normal Reporter: Bogdan Irimie Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5058 Fix sync

Change By: Bogdan Irimie Story Points: 3

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5058 Fix sync

Change By: Bogdan Irimie Sprint: ghost-24.03.2021

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie assigned an issue to Sebastian Miclea

PuppetDB / PDB-5058 Fix sync

Change By: Bogdan Irimie Assignee: Sebastian Miclea

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5058 Fix sync

Change By: Bogdan Irimie Acceptance Criteria: * sync no longer alters the reports it pulls from the other side

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5058 Fix sync

Change By: Bogdan Irimie Acceptance Criteria:

* sync no longer alters the reports it pulls from the other side

* sync unit test which checks the point above

Add Comment

[Bogdan Irimie (Jira)]

Bogdan Irimie updated an issue

PuppetDB / PDB-5058 Fix sync Change By: Bogdan Irimie

pdbext sync has a filter in the [:clean-up-record-fn|https://github.com/puppetlabs/pe-puppetdb-extensions/blob/6.x/src/puppetlabs/pe_puppetdb_extensions/sync/core.clj#L173-L179] for reports which removes any *resource_events* which would be expired locally. This filter was added to account for differing ttls between *reports* and *resource_events*. We wanted to avoid a situation where GC would clean up a *resource_event* partition and then sync would pull a report with *resource_events* that recreated the deleted partition. As a result if an *event* is pulled out of a *resource* in a *report* we could have reports which don't exactly match between two pdbs syncing with one another.   This problem could become more pronounced when we add the ability to disable *resource_event* storage in PDB-3653. If the *resource-event-ttl* is set to 0 the sync filter will strip out all *resource_events* in the reports it transfers. It would be better if the check for expired *resource_events* was moved into the *report* ingestion code. That way sync would keep the report identical on both sides.   To do this we would need to remove the filter in the [:clean-up-record-fn|https://github.com/puppetlabs/pe-puppetdb-extensions/blob/6.x/src/puppetlabs/pe_puppetdb_extensions/sync/core.clj#L173-L179] for reports and adjust at least the [dont-pull-events-that-would-be-expired-locally|https://github.com/puppetlabs/pe-puppetdb-extensions/blob/6.x/test/puppetlabs/pe_puppetdb_extensions/sync/end_to_end_test.clj#L939] test to check that the *resource-events* partitions aren't created not that the events don't exist in the report body.

Add Comment