Jira (PUP-10675) ensure pluginsync is atomic

16 views
Skip to first unread message

Ciprian Badescu (Jira)

unread,
Sep 16, 2020, 8:14:04 AM9/16/20
to puppe...@googlegroups.com
Ciprian Badescu created an issue
 
Puppet / Bug PUP-10675
ensure pluginsync is atomic
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2020/09/16 5:13 AM
Priority: Normal Normal
Reporter: Ciprian Badescu

There are various scenarios in which catalog and plugins may not match:

  • plugins are sync-ed while new catalog cannot be retrieved and cached catalog is used instead
  • plugins are partially sync-ed, using new or cached catalog may both lead to inconsistencies

To fix this plugins need to be downloaded in an atomic way. One option is to create a secure directory with the same parent as Puppet[:libdir] and use it to pluginsync. To avoid downloading files we already have, you'd have to copy everything from Puppet[:libdir] to libnew. So maybe something like:

1. create lib.new using ruby equivalent of mktemp -d lib.new and restrict permissions so only the current user can write
2. Pluginsync to lib.new
3. Rename Puppet[:libdir] to lib.old
4. Rename lib.new to Puppet[:libdir]
5. Delete lib.old recursively

It's possible for puppet to crash/ctrl-c between any of those points. If it occurs between 3 and 4, then puppet will have lost all of its plugins. To account for that, if puppet starts and lib.old exists, but Puppet[:libdir] doesn't, then have recover its plugins by renaming lib.old to Puppet[:libdir].

If the crash occurs between 4 and 5, then the new plugins have been committed. So when puppet starts, if Puppet[:libdir] exists, have it delete the stale lib.old.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Ciprian Badescu (Jira)

unread,
Sep 16, 2020, 8:15:04 AM9/16/20
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
There are various scenarios in which catalog and plugins may not match:
- plugins are sync-ed while new catalog cannot be retrieved and cached catalog is used instead (this should be subject of https://tickets.puppetlabs.com/browse/PUP-10648 and https://tickets.puppetlabs.com/browse/PUP-10667)
- plugins are partially sync-ed, using new or cached catalog may both lead to inconsistencies


To fix this plugins need to be downloaded in an atomic way. One option is to create a secure directory with the same parent as Puppet[:libdir] and use it to pluginsync. To avoid downloading files we already have, you'd have to copy everything from Puppet[:libdir] to libnew. So maybe something like:

1. create lib.new using ruby equivalent of mktemp -d lib.new and restrict permissions so only the current user can write
2. Pluginsync to lib.new
3. Rename Puppet[:libdir] to lib.old
4. Rename lib.new to Puppet[:libdir]
5. Delete lib.old recursively

It's possible for puppet to crash/ctrl-c between any of those points. If it occurs between 3 and 4, then puppet will have lost all of its plugins. To account for that, if puppet starts and lib.old exists, but Puppet[:libdir] doesn't, then have recover its plugins by renaming lib.old to Puppet[:libdir].

If the crash occurs between 4 and 5, then the new plugins have been committed. So when puppet starts, if Puppet[:libdir] exists, have it delete the stale lib.old.

Josh Cooper (Jira)

unread,
Sep 16, 2020, 5:39:03 PM9/16/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Night's Watch

Mihai Buzgau (Jira)

unread,
Sep 23, 2020, 4:08:04 AM9/23/20
to puppe...@googlegroups.com
Mihai Buzgau updated an issue
Change By: Mihai Buzgau
Sprint: PR - Triage

Mihai Buzgau (Jira)

unread,
Sep 23, 2020, 4:13:03 AM9/23/20
to puppe...@googlegroups.com
Mihai Buzgau updated an issue
Change By: Mihai Buzgau
Story Points: 5

Bogdan Irimie (Jira)

unread,
Nov 5, 2020, 3:51:03 AM11/5/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Nov 5, 2020, 3:52:05 AM11/5/20
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ready for triage

Ciprian Badescu (Jira)

unread,
Jun 22, 2021, 3:51:02 AM6/22/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Sprint: ready for triage
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Ciprian Badescu (Jira)

unread,
Jun 22, 2021, 3:52:01 AM6/22/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Epic Link: PUP-11032
Reply all
Reply to author
Forward
0 new messages