Jira (PUP-3922) Pluginsync collisions

[redmine.exporter (JIRA)]

redmine.exporter created an issue

Puppet / PUP-3922 Pluginsync collisions Issue Type: Bug Assignee: Unassigned Created: 2015/01/28 3:21 PM Labels: redmine Priority: Normal Reporter: redmine.exporter I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc. Add Comment

This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a)

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922 Pluginsync collisions

Change By: Eric Sorenson Reporter: redmine.exporter Eric Sorenson

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922

Preserve modulepath structure during pluginsync Change By: Eric Sorenson Summary: Pluginsync collisions Preserve modulepath structure during pluginsync

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync Change By: Eric Sorenson

Original problem report from [~ben.ford]: {quote}

I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc.

{quote} My comment from two years ago: {quote} The namespacing/load order is problematic but the files being copied on top of one another would be fixed if we replicated the module path structure from the master onto the client instead of squashing everything into $plugindest regardless of source. {quote} It turns out that this problem blocked our ability to execute on [PUP-1077], because newer versions of a module, installed via pluginsync, which restructured files would incompletely overlay

Add Comment

[Eric Sorenson (JIRA)]

It turns out that this problem blocked our ability to execute on [PUP-1077], because newer versions of a module, installed via pluginsync, which restructured files would incompletely overlay    the same module structure included in a package's modulepath.

Add Comment

[Eric Sorenson (JIRA)]

If we instead preserve the modulepath structure that the master discovers as it builds a file set for pluginsync and send the lib/ directories down to the agent with their module name prepended,

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync Change By: Eric Sorenson

Original problem report from [~ben.ford]: {quote} I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc. {quote} My comment from two years ago: {quote} The namespacing/load order is problematic but the files being copied on top of one another would be fixed if we replicated the module path structure from the master onto the client instead of squashing everything into $plugindest regardless of source. {quote}

It turns out that this problem blocked our ability to execute on [PUP-1077], because newer versions of a module , installed via pluginsync,  which restructured files would , when pluginsync'ed,  incompletely overlay the same module structure included in a package's modulepath. If we instead preserve the modulepath structure that the master discovers as it builds a file set for pluginsync and send the lib/ directories down to the agent with their module name prepended,    the agent could detect that there was an overlapping module name and exclude the built-in one from its load path.

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Eric Sorenson Fix Version/s: PUP 4.x

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Eric Sorenson Priority: Normal Major

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-3922

Re: Preserve modulepath structure during pluginsync Josh Cooper please keep me honest by correcting the description if I've misconstrued the issue or omitted anything.

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson updated an issue

Puppet / PUP-3922

Preserve modulepath structure during pluginsync

Change By: Eric Sorenson

Original problem report from [~ben.ford]: {quote} I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc. {quote} My comment from two years ago: {quote} The namespacing/load order is problematic but the files being copied on top of one another would be fixed if we replicated the module path structure from the master onto the client instead of squashing everything into $plugindest regardless of source. {quote}

It turns out that this problem blocked our ability to execute on [PUP-1077]  and any related efforts to modularize things currently in Puppet core , because newer versions of a module which restructured files would, when pluginsync'ed, incompletely overlay the same module structure included in a package's modulepath.

If we instead preserve the modulepath structure that the master discovers as it builds a file set for pluginsync and send the lib/ directories down to the agent with their module name prepended, the agent could detect that there was an overlapping module name and exclude the built-in one from its load path.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Josh Cooper

Original problem report from [~ben.ford]: {quote} I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc. {quote} My comment from two years ago: {quote} The namespacing/load order is problematic but the files being copied on top of one another would be fixed if we replicated the module path structure from the master onto the client instead of squashing everything into $plugindest regardless of source. {quote}

It turns out that this problem blocked our ability to execute on [PUP-1077] and any related efforts to modularize things currently in Puppet core, because newer versions of a module which restructured files would, when pluginsync'ed, incompletely overlay the same module structure included in a package's modulepath. If we instead preserve the modulepath structure that the master discovers as it builds a file set for pluginsync and send the lib/ directories down to the agent with their module name prepended, the agent could detect that there was an overlapping module name and exclude the built-in one from its load path.  This would ensure that the agent would only have one version of a module in its $LOAD_PATH. If we allow multiple versions, then we will reintroduce problems like redmine 7316, caused by having multiple versions of code in the same

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync Change By: Josh Cooper

Original problem report from [~ben.ford]: {quote} I had a problem come up in training last week involving synced facts that brought out a larger problem in pluginsync. So if I create a module 'one' and define a fact in it called 'myfact', then create a module 'two' and also define a fact there called 'myfact' they will collide. The fact in 'two' will win because it comes later in the directory globbing. But worse is that they will be continuously fighting. On an agent run you can see this fact being overwritten to 'one/myfact' and then 'two/myfact' every single time. This is horrible for usability because it's not clear which fact will be called, and it's a terrible waste of bandwidth, etc. {quote} My comment from two years ago: {quote} The namespacing/load order is problematic but the files being copied on top of one another would be fixed if we replicated the module path structure from the master onto the client instead of squashing everything into $plugindest regardless of source. {quote} It turns out that this problem blocked our ability to execute on [PUP-1077] and any related efforts to modularize things currently in Puppet core, because newer versions of a module which restructured files would, when pluginsync'ed, incompletely overlay the same module structure included in a package's modulepath.

If we instead preserve the modulepath structure that the master discovers as it builds a file set for pluginsync and send the lib/ directories down to the agent with their module name prepended, the agent could detect that there was an overlapping module name and exclude the built-in one from its load path. This would ensure that the agent would only have one version of a module in its $LOAD_PATH. If we allow multiple versions  of code in the $LOAD_PATH , then we will reintroduce problems like redmine 7316 , caused by having multiple versions of code in the same  .

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Josh Cooper Scrum Team: Client Platform Story Points: 3

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Kylo Ginsberg Sprint: Client 2015-06-24

Add Comment

This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)

[Steve Barlow (JIRA)]

Steve Barlow updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Steve Barlow Sprint: Client 2015- 06 07 - 24 08

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Josh Cooper Sprint: Client 2015-07- 08 22

Add Comment

[Kylo Ginsberg (JIRA)]

Kylo Ginsberg updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Kylo Ginsberg Sprint: Client 2015-07-22

Add Comment

This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)

[William Hopper (JIRA)]

William Hopper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: William Hopper Story Points: 3

Add Comment

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)

[Kenn Hussey (JIRA)]

Kenn Hussey updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Kenn Hussey Fix Version/s: PUP 4.y Fix Version/s: PUP 5.y

Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Geoff Nichols Team: Agent

Add Comment

[Sean McDonald (JIRA)]

Sean McDonald updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Sean McDonald Labels: redmine  triaged

Add Comment

[Moses Mendoza (JIRA)]

Moses Mendoza updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Moses Mendoza Labels: redmine  triaged

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-3922 Preserve modulepath structure during pluginsync

Change By: Josh Cooper Sub-team: Coremunity

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-3922

Re: Preserve modulepath structure during pluginsync I no longer think this issue is a blocker for removing non-core types and providers. But there are still valid reasons for preserving module namespace during pluginsync, so I'll keep this issue open, but remove it from the epic.

Add Comment

[Trevor Vaughan (Jira)]

Trevor Vaughan commented on PUP-3922

Re: Preserve modulepath structure during pluginsync

Issues with things like client side namespacing have been problematic in the module space for years. Honestly, I think that everything should be namespaced at this point and that it should all be completely unambiguous where possible. In my opinion, one of the main issues is that this causes users to try modules form the Forge but either reject them, or fork them, due to namespace collisions. Fixing this issue would be a boon for the overall reusability of the module space in general.

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-3922

Preserve modulepath structure during pluginsync

Change By: Josh Cooper Epic Link: PUP-11032

Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

[Jesse Hathaway (Jira)]

Jesse Hathaway commented on PUP-3922

Re: Preserve modulepath structure during pluginsync Has this situation been improved since Puppet 5.5, the version we are currently running?

Add Comment

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-3922

Re: Preserve modulepath structure during pluginsync

lollipopman Sadly no changes since 5.5.

Add Comment