Jira (BOLT-1318) Modulepath unable to handle folder names in uppercase characters on Windows

0 views
Skip to first unread message

Kevin Reeuwijk (JIRA)

unread,
May 17, 2019, 5:56:03 PM5/17/19
to puppe...@googlegroups.com
Kevin Reeuwijk created an issue
 
Puppet Task Runner / Bug BOLT-1318
Modulepath unable to handle folder names in uppercase characters on Windows
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2019/05/17 2:55 PM
Environment:

User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything

Priority: Normal Normal
Reporter: Kevin Reeuwijk

Bolt won’t handle folder names with one or more uppercase characters in the name incorrectly, when they are part of the modulepath. I encountered this while doing a Bolt Workshop at KPN.

If the modulepath has a capitalized foldername in it, not Tasks or Plans are found in that module. It is as if the module is not there.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Alex Dreyer (JIRA)

unread,
May 17, 2019, 6:01:03 PM5/17/19
to puppe...@googlegroups.com

Alex Dreyer (JIRA)

unread,
May 28, 2019, 5:15:04 PM5/28/19
to puppe...@googlegroups.com
Alex Dreyer updated an issue
Bolt won’t handle folder names with one or more uppercase characters in the name incorrectly, when they are part of the modulepath. I encountered this while doing a Bolt Workshop at KPN.

If the modulepath has a capitalized foldername in it, not Tasks or Plans are found in that module. It is as if the module is not there.

User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything

Alex Dreyer (JIRA)

unread,
May 28, 2019, 5:15:04 PM5/28/19
to puppe...@googlegroups.com

User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything

Alex Dreyer (JIRA)

unread,
May 28, 2019, 5:15:35 PM5/28/19
to puppe...@googlegroups.com
Alex Dreyer updated an issue
Change By: Alex Dreyer
Comment:
Is this about part of the module path

User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything

Lucy Wyman (JIRA)

unread,
Jun 18, 2019, 12:50:03 PM6/18/19
to puppe...@googlegroups.com

Tom Beech (JIRA)

unread,
Jun 19, 2019, 6:35:03 PM6/19/19
to puppe...@googlegroups.com

Tom Beech (JIRA)

unread,
Jun 20, 2019, 6:13:04 PM6/20/19
to puppe...@googlegroups.com
Tom Beech commented on Bug BOLT-1318
 
Re: Modulepath unable to handle folder names in uppercase characters on Windows

This doesn't seem to be an issue with Bolt, but Pal instead. Bolt only reads the modulepath from the command line and then passes it as a string to Pal, which is responsible for compiling a list of all plans at the modulepath.

 

We should discuss whether or not this is expected behavior and if the modulepath should be case sensitive or not on Windows.

Henrik Lindberg (JIRA)

unread,
Jun 21, 2019, 4:28:03 AM6/21/19
to puppe...@googlegroups.com

PAL in turn uses loaders to find what is loadable. Loaders performs a glob of all possibly loadable things (once), and then when there is a request to load, it computes possible candidates to load based on a) what type/name is requested, and b) the result from the glob. The production of candidates is probably where the support for case independent compare needs to be made. Adding logic there will have an impact on performance as loading is time critical and used a lot.

Tom Beech (JIRA)

unread,
Jun 21, 2019, 12:53:04 PM6/21/19
to puppe...@googlegroups.com
Tom Beech commented on Bug BOLT-1318

The original ticket is a little unclear, but here's what the issue is:

 

Given the module path C:\DATA\workshop with a plan named foo:

 

If the user enters the command bolt plan show --modulepath c:/data/workshop then foo will not be in the list of plans.

If the user enters the command bolt plan show --modulepath C:/DATA/workshop then foo will be in the list of plans.

 

When using --modulepath, the specified path must match the case of the actual module path. Whether or not the actual path has uppercase characters won't affect Bolt's ability to locate the module.

Henrik Lindberg (JIRA)

unread,
Jun 21, 2019, 1:46:03 PM6/21/19
to puppe...@googlegroups.com

Kevin Reeuwijk (JIRA)

unread,
Jun 24, 2019, 9:03:05 AM6/24/19
to puppe...@googlegroups.com

except that the auto-generated modulepath is all lowercase, see the original ticket info:

```User has this Bolt project folder:


`C:\DATA\workshop` with a bolt.yaml in it

and subsequently the modulepath becomes c:/data/workshop/modules```

Notice that the modulepath didn't become `c:/DATA/workshop/modules`, but `c:/data/workshop/modules`

So a quick fix could be to correctly auto-generate the modulepath so that it honors the source case of characters.

Cas Donoghue (JIRA)

unread,
Jun 24, 2019, 1:07:04 PM6/24/19
to puppe...@googlegroups.com
Cas Donoghue commented on Bug BOLT-1318

Kevin Reeuwijk I'm having trouble reproducing the case you mention (vmpooler win-10-pro)

PS C:\DATA> New-Item -Type File -Path bolt.yaml -Force
 
 
    Directory: C:\DATA
 
 
Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        6/24/2019   5:03 PM              0 bolt.yaml
 
 
PS C:\DATA> ls
 
 
    Directory: C:\DATA
 
 
Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        6/24/2019   4:57 PM                modules
-a----        6/24/2019   4:59 PM             43 .rerun.json
-a----        6/24/2019   5:03 PM              0 bolt.yaml
 
 
PS C:\DATA> cat .\modules\test\tasks\init.ps1
Write-Host 'hi'
PS C:\DATA> bolt task show
facts                    Gather system facts
package                  Manage and inspect the state of packages
puppet_agent::install    Install the Puppet agent package
puppet_agent::version    Get the version of the Puppet agent package installed. Returns nothing if none present.
puppet_conf              Inspect puppet agent configuration settings
reboot                   Reboots a machine
reboot::last_boot_time   Gets the last boot time of a Linux or Windows system
service                  Manage and inspect the state of services
test
 
MODULEPATH:
C:/DATA/modules:C:/DATA/site-modules:C:/DATA/site
 
Use `bolt task show <task-name>` to view details and parameters for a specific task.
PS C:\DATA> bolt task run test -t localhost
Started on localhost...
Finished on localhost:
  hi
  {
  }
Successful on 1 node: localhost
Ran on 1 node in 0.69 seconds

could it be platform specific? Or is there a problem with my repro case?

Kevin Reeuwijk (JIRA)

unread,
Jun 24, 2019, 1:37:03 PM6/24/19
to puppe...@googlegroups.com

Cas Donoghue is that with the latest Bolt version? Because it seems this issue is then fixed in the latest version...

This output is encouraging, as it was shown in all lower case in an older Bolt version:

MODULEPATH:
C:/DATA/modules:C:/DATA/site-modules:C:/DATA/site

Cas Donoghue (JIRA)

unread,
Jun 24, 2019, 1:50:03 PM6/24/19
to puppe...@googlegroups.com

Kevin Reeuwijk (JIRA)

unread,
Jun 25, 2019, 5:00:04 AM6/25/19
to puppe...@googlegroups.com

I checked again with a couple of Bolt versions:

  • Bolt 1.10 - no issue
  • Bolt 1.20 - no issue
  • Bolt 1.24 - no issue

All three were able to correctly find the module when the modulepath is auto-generated.

The user encountering the problem must have manually set the modulepath in bolt.yaml and used a lowercase path while the actual path container uppercase characters.

So officially this is more user error than bug. But having said that, Windows users expect case insensitivity.
We may want to evaluate handling the modulepath as specified in bolt.yaml as case insensitive on Windows specifically.

Alex Dreyer (JIRA)

unread,
Jun 25, 2019, 10:35:03 AM6/25/19
to puppe...@googlegroups.com
Alex Dreyer commented on Bug BOLT-1318

Making modulepath case insensitive on windows would make project directories incompatible across platforms which seems worse. Also modules, tasks and the rest of the puppet objects won't be so it seems better for users to learn that puppet is case sensitive early.

We may want to consider checking and warning when a path only exists in case sensitive form.

Lucy Wyman (JIRA)

unread,
Jun 25, 2019, 12:16:03 PM6/25/19
to puppe...@googlegroups.com
Lucy Wyman updated an issue
Change By: Lucy Wyman
Bolt won’t handle folder names with one or more uppercase characters in the name incorrectly, when they are part of the modulepath. I encountered this while doing a Bolt Workshop at KPN.

If the modulepath has a capitalized foldername in it, not Tasks or Plans are found in that module. It is as if the module is not there.


User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything


Update:
- When on windows, we should check to see that all paths in the modulepath exist using Ruby's File tooling, and if they don't exist we should check for a case-insensitive version and warn if that exists. The warning should state that modulepaths are case-sensitive.

Lucy Wyman (JIRA)

unread,
Jun 25, 2019, 12:17:04 PM6/25/19
to puppe...@googlegroups.com
Lucy Wyman updated an issue
Bolt won’t handle folder names with one or more uppercase characters in the name incorrectly, when they are part of the modulepath. I encountered this while doing a Bolt Workshop at KPN.

If the modulepath has a capitalized foldername in it, not Tasks or Plans are found in that module. It is as if the module is not there.


User has this Bolt project folder:
`C:\DATA\workshop` with a bolt.yaml in it
and subsequently the modulepath becomes c:/data/workshop/modules

However no tasks/plans are ever found there.
If I manually force the modulepath (in bolt.yaml) to C:/DATA/workshop then it works
Bolt needs to be completely case insensitive on Windows, otherwise this breaks everything

Update:
- When on windows, we should check to see that all paths in the modulepath exist using Ruby's File tooling the canonicalized filename , and if they don't exist we should check for a case-insensitive version and warn if that exists. The warning should state that modulepaths are case-sensitive.

Tom Beech (JIRA)

unread,
Jun 25, 2019, 8:10:04 PM6/25/19
to puppe...@googlegroups.com
Tom Beech commented on Bug BOLT-1318
 
Re: Modulepath unable to handle folder names in uppercase characters on Windows

Digging around a bit more it looks like all platforms are case-sensitive, not just windows. This holds wherever the user provides a path, whether at the command line when using --boltdir or --modulepath, in or in a bolt.yaml.  Providing an error like this should be easy enough:

 

$ bolt plan show --modulepath c:/data/workshop

  WARNING: Bolt is case sensitive when providing a module path. Did you mean:{{              
}}           C:/DATA/workshop

 

Since paths are case-sensitive across platforms would it be preferable to have this type of warning, stress the case-sensitivity in the docs, or both?

Kevin Reeuwijk (JIRA)

unread,
Jun 26, 2019, 4:49:03 AM6/26/19
to puppe...@googlegroups.com

Alex Dreyer (JIRA)

unread,
Jun 26, 2019, 12:36:03 PM6/26/19
to puppe...@googlegroups.com
Alex Dreyer commented on Bug BOLT-1318

I think a note or warning in the modulepath docuementation makes sense too.

Tom Beech (JIRA)

unread,
Jul 1, 2019, 3:33:03 PM7/1/19
to puppe...@googlegroups.com

Michelle Fredette (JIRA)

unread,
Jul 10, 2019, 2:18:04 PM7/10/19
to puppe...@googlegroups.com

Michelle Fredette (JIRA)

unread,
Jul 10, 2019, 2:19:02 PM7/10/19
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages