Jira (PUP-11106) superclass mismatch for class Uniquefile (TypeError)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError) Change By: Josh Cooper Affects Version/s: PUP 7.7.0 Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

Josh Cooper moved an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError)

Change By: Josh Cooper Key: PA PUP - 3828 11106 Affects Version/s: puppet-agent 7.7.0 Project: Puppet Agent

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError) Philipp H it seems you have two versions of puppet installed. One is in /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet, the other is in /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet. The latter is not expected (the puppet-agent package installs under /opt/puppetlabs. Can you check your environment and ruby load path to see if you have conflicts?

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Josh Cooper Thank you very much for your input! As I am using atomic updates ( fedora coreos ), the folders in /opt are mounted from /usr/lib/opt and puppet is actually installed in /usr/lib/opt. This was also the case in version 7.6.1. As you mentioned there seems to be a load path conflict with this setup in the new version. Is there any way to force the puppet ruby load path? (I guess I have to figure this out myself, as this version of fedora is not really supported.)

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

It sounds like maybe you're using packages from someone other than Puppet. Can you try nightly builds of puppet-agent 7.x (http://nightlies.puppet.com/yum/). We should have official packages for Fedora 34 out real soon.

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Josh Cooper I've switched to the nightly repository and installed puppet-agent-7.8.0.72.g80c5de1e1-1.fc34.x86_64 I've always used the official rpms provided by puppet. (using the platform repository: https://puppet.com/docs/puppet/7/install_puppet.html#enable_the_puppet_platform_repository)   Due to the atomic nature of fedora coreos everthing installed under /opt is moved to the /usr/lib/opt and then linked. (I know it is kind of weird. I am also checking if it possible to prevent this move.) lrwxrwxrwx. 1 root root 23 Jun 10 10:06 puppetlabs -> /usr/lib/opt/puppetlabs     I am not sure what changed since version 7.6.1-1 that it tries to load the ruby files in /opt/puppetlabs and /usr/lib/opt/puppetlabs. Is there any way from "outside" (like the puppet.conf or some ruby args? ) to prevent it from loading the files twice or ignoring /usr/lib/opt in the first place?

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

I have downgraded to puppet-agent-7.6.1-1.fc32.x86_64.rpm for now.

Add Comment

[Philipp H (Jira)]

Philipp H updated an issue

Puppet / PUP-11106

superclass mismatch for class Uniquefile (TypeError)

Change By: Philipp H Affects Version/s: PUP 7.8.0

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError) Can I help you in any way? Is there any more information I can provide you?

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Philipp H could you append --logdest /tmp/agent.log, delete any sensitive info, and attach that file to the ticket?

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Philipp H Ok, so the UniqueFile class is somewhat special, as it will result in a hard failure if an attempt is made to load the file twice. The Uniquefile class extends an anonymous superclass Delegate(File). So the second time Uniquefile is loaded, the file will reopen the class and try to assign it to a different superclass, which is not allowed. My guess is the combination of symlinks and relative_require is causing uniquefile.rb to first be loaded as: /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/uniquefile.rb Then the autoloader calls Dir.glob so we see: 8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/file_metadata/selector.rb:1:in `<top (required)>' And that ends trying to load uniquefile with a different path: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/uniquefile.rb Prior to the move to relative_require, puppet would have searched its $LOAD_PATH for something matching puppet/file_system/uniquefile.rb, and /usr/lib/opt/... most likely was earlier in the search path. Note there is a closely related problem on Windows, which results in the same exception when mixing 8.3 and long file paths, see PUP-11184.

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Josh Cooper If I append the "logdest" no log is created. Actually I can pass any argument to the executable and I get the same traceback/errors as mentioned above. Seems like it breaks before even parsing the arguments. Thanks for the explanation. I guess that explains why it stopped working from the 7.6.1 onward. If I understood you correctly, there is no way to force the require path from "outside", is there?

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

I was able to reproduce on Fedora 34 CoreOS. I replicated the symlink structure on Ubuntu 18.04 and I receive the same error: # ls -l /opt lrwxrwxrwx 1 root root 8 Aug 17 05:32 /opt -> /var/opt # ls -l /var/opt total 0 lrwxrwxrwx 1 root root 23 Aug 17 05:31 puppetlabs -> /usr/lib/opt/puppetlabs # ls -l /usr/lib/opt/puppetlabs total 16 drwxr-xr-x 2 root root 4096 Aug 17 05:16 bin drwxr-xr-x 3 root root 4096 Aug 17 05:16 facter drwxr-xr-x 11 root root 4096 Aug 17 05:16 puppet drwxr-xr-x 5 root root 4096 Aug 17 05:16 pxp-agent # /opt/puppetlabs/puppet/bin/puppet --help ... 38: from /opt/puppetlabs/puppet/bin/puppet:4:in `<main>' 37: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require' 36: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require' 35: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:30:in `<top (required)>' 34: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:30:in `require_relative' 33: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:42:in `<top (required)>' 32: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:340:in `<module:Puppet>' 31: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:340:in `require_relative' 30: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/runtime.rb:1:in `<top (required)>' 29: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/runtime.rb:1:in `require_relative' 28: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http.rb:1:in `<top (required)>' 27: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http.rb:9:in `<module:Puppet>' 26: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http.rb:26:in `<module:HTTP>' 25: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http.rb:26:in `require_relative' 24: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http/service/file_server.rb:1:in `<top (required)>' 23: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/http/service/file_server.rb:1:in `require_relative' 22: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_serving/metadata.rb:9:in `<top (required)>' 21: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_serving/metadata.rb:14:in `<class:Metadata>' 20: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector.rb:49:in `indirects' 19: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector.rb:49:in `new' 18: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:110:in `initialize' 17: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:121:in `set_global_setting' 16: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:180:in `validate_terminus_class' 15: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/terminus.rb:112:in `terminus_class' 14: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:49:in `loaded_instance' 13: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/concurrent/lock.rb:10:in `synchronize' 12: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:54:in `block in loaded_instance' 11: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:183:in `load' 10: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:79:in `load_file' 9: from /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:79:in `load'

8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/file_metadata/selector.rb:1:in `<top (required)>'

7: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/file_metadata/selector.rb:1:in `require_relative' 6: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_serving/metadata.rb:1:in `<top (required)>' 5: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_serving/metadata.rb:1:in `require_relative' 4: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:17:in `<top (required)>' 3: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:17:in `require_relative' 2: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:11:in `<top (required)>' 1: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:11:in `require_relative' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/uniquefile.rb:17:in `<top (required)>': superclass mismatch for class Uniquefile (TypeError) The issue occurs because ruby stores entries in the $LOADED_FEATURES hash using their realpath, for example: # /opt/puppetlabs/puppet/bin/ruby -e 'module Puppet; end; require "puppet/file_system/uniquefile"; puts $LOADED_FEATURES.grep(/unique/)' /usr/lib/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/file_system/uniquefile.rb However, when the autoloader tries to load the selector terminus, it does so by joining each directory entry from $LOAD_PATH with 'puppet/indirector/file_metadata/selector.rb', and loading the first file that exists. But Puppet's vendored ruby hardcodes the prefix as '/opt/puppetlabs/puppet/lib/ruby/vendor_ruby', so we end up calling: Kernel.load '/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/file_metadata/selector.rb' And since that path doesn't match what's in $LOADED_FEATURES, we try to load the same class from two files, instead of reloading the existing class. The autoloader really should be storing the realpath in $LOADED_FEATURES. That said, I'm curious how you're using puppet on CoreOS given the host filesystem is immutable?

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Josh Cooper /etc and /var is mounted as read-write in CoreOs. Traditional places that might hold state (e.g. /home, or /srv) are symlinks to directories in /var (e.g. /var/home or /var/srv).  Further Info on the Disk Layout: https://docs.fedoraproject.org/en-US/fedora-coreos/storage/#_disk_layout The Puppet Agent is actually running fine if I just move the Cache directory (vardir) (puppet.conf or argument) ** for example to /var/.puppetcache (default is /opt/puppetlabs/puppet/cache which is mounted ro).

Add Comment

[Philipp H (Jira)]

Philipp H commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

Josh Cooper In hindsight It might not have been the smartest "move" to use puppet on this system. (Actually I wanted to run everything, also the puppet agent in a container (podman), but I had some issues there with kernel modules, for example if I want to use the firewall module and manage iptables.)

Add Comment

[Josh Cooper (Jira)]

Josh Cooper assigned an issue to Josh Cooper

Puppet / PUP-11106

superclass mismatch for class Uniquefile (TypeError)

Change By: Josh Cooper Assignee: Josh Cooper

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError)

Change By: Josh Cooper Team: Coremunity

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError)

Change By: Josh Cooper Sprint: Coremunity Kanban

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError)

Change By: Josh Cooper Fix Version/s: PUP 7.11.0

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Puppet / PUP-11106 superclass mismatch for class Uniquefile (TypeError)

Change By: Ciprian Badescu Fix Version/s: PUP 7.11.0 Fix Version/s: PUP 7.12.0

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError) So there's an easy way to reproduce the issue: # mkdir -p /tmp/realpath/lib # cat <<END > /tmp/realpath/lib/a.rb require_relative 'b' puts load('/tmp/link/lib/b.rb') END # cat <<END > /tmp/realpath/lib/b.rb require_relative 'c' END # cat <<END > /tmp/realpath/lib/c.rb require 'tempfile' class C < DelegateClass(File); end END # ln -s /tmp/realpath/ /tmp/link The following works using system ruby: # ruby /tmp/link/lib/a.rb true But using our vendored ruby does not: # /opt/puppetlabs/puppet/bin/ruby /tmp/link/lib/a.rb Traceback (most recent call last): 4: from /tmp/link/lib/a.rb:2:in `<main>' 3: from /tmp/link/lib/a.rb:2:in `load' 2: from /tmp/link/lib/b.rb:1:in `<top (required)>' 1: from /tmp/link/lib/b.rb:1:in `require_relative' /tmp/link/lib/c.rb:2:in `<top (required)>': superclass mismatch for class C (TypeError) System ruby does the following: # strace -f -e trace=file ruby /tmp/link/lib/a.rb ... [pid 1444] openat(AT_FDCWD, "/tmp/link/lib/b.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7 [pid 1444] openat(AT_FDCWD, "/tmp/link/lib/b.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 7 [pid 1444] lstat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0 [pid 1444] lstat("/tmp/link", {st_mode=S_IFLNK|0777, st_size=14, ...}) = 0 [pid 1444] readlink("/tmp/link", "/tmp/realpath/", 100) = 14 [pid 1444] lstat("/tmp/realpath", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 [pid 1444] lstat("/tmp/realpath/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 [pid 1444] lstat("/tmp/realpath/lib/b.rb", {st_mode=S_IFREG|0644, st_size=21, ...}) = 0 true While ours does: # strace -f -e trace=file /opt/puppetlabs/puppet/bin/ruby /tmp/link/lib/a.rb ... openat(AT_FDCWD, "/tmp/link/lib/b.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5 openat(AT_FDCWD, "/tmp/link/lib/b.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5 openat(AT_FDCWD, "/tmp/link/lib/c.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5 openat(AT_FDCWD, "/tmp/link/lib/c.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5 Traceback (most recent call last): 4: from /tmp/link/lib/a.rb:2:in `<main>' 3: from /tmp/link/lib/a.rb:2:in `load' 2: from /tmp/link/lib/b.rb:1:in `<top (required)>' 1: from /tmp/link/lib/b.rb:1:in `require_relative' /tmp/link/lib/c.rb:2:in `<top (required)>': superclass mismatch for class C (TypeError) The behavior difference is due to our ruby patch added in PA-3526, as we no longer call File.realpath: https://github.com/puppetlabs/puppet-runtime/blob/master/resources/patches/ruby_27/ruby-faster-load_27.patch

Add Comment

[Sebastian (Jira)]

Sebastian commented on PUP-11106

Re: superclass mismatch for class Uniquefile (TypeError)

We currently have an ZFS server which runs into the same error since /opt is a symlink to a zfs pool: lrwxrwxrwx. 1 root root 8 2. Mär 2017 opt -> tank/opt Can i provide further information to resolve this issue? Greetings

Add Comment