Jira (PUP-11476) Puppet code deploys fail with 2019.8.10 on RHEL 7.9

8 views
Skip to first unread message

Paul Riley (Jira)

unread,
Mar 9, 2022, 9:16:01 AM3/9/22
to puppe...@googlegroups.com
Paul Riley created an issue
 
Puppet / Bug PUP-11476
Puppet code deploys fail with 2019.8.10 on RHEL 7.9
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2022/03/09 6:15 AM
Priority: Normal Normal
Reporter: Paul Riley

*Puppet Version: 6.26.0
*Puppet Server Version: 2019.8.10
*OS Name/Version: RedHat 7.9

Describe your issue in as much detail as possible…

When running code deploy on a freshly deployed RedHat 7.10 system, it fails with the following error below after running the following commands after initially provisioning the system.

This is occurring when running puppet code deploy <any param> or sudo -H -u pe-puppet bash -C \ '/opt/puppetlabs/puppet/bin/r10k deploy environment -c /opt/puppetlabs/server/data/code-manager/r10k.yaml -p -v debug'

[2022-03-08 10:08:52 - DEBUG] Using token from license to connect to the Forge.
[2022-03-08 10:08:52 - DEBUG] Fetching 'git@<redacted>:nonprod-pe/nonprod_pe_control_repo.git' to determine current branches.

      • Error in `/opt/puppetlabs/puppet/bin/ruby': corrupted size vs. prev_size: 0x0000000003716a80 ***
        ======= Backtrace: =========
        /lib64/libc.so.6(+0x7f474)[0x7fa1317ee474]
        /lib64/libc.so.6(+0x82c84)[0x7fa1317f1c84]
        /lib64/libc.so.6(__libc_malloc+0x4c)[0x7fa1317f478c]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0xab1d3)[0x7fa1325411d3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x2f861)[0x7fa1324c5861]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x34eb3)[0x7fa1324caeb3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_ary_push+0x29)[0x7fa1324caf79]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x172390)[0x7fa132608390]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1acef6)[0x7fa132642ef6]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1c88)[0x7fa132647c88]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bd5ec)[0x7fa1326535ec]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_funcallv+0xdd)[0x7fa132655d2d]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0xeb321)[0x7fa132581321]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1acef6)[0x7fa132642ef6]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1c88)[0x7fa132647c88]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bd5ec)[0x7fa1326535ec]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_funcallv+0xdd)[0x7fa132655d2d]
        /opt/puppetlabs/puppet/lib/ruby/gems/2.5.0/gems/rugged-0.27.7/lib/rugged/rugged.so(+0x1e38f)[0x7fa1297b038f]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_protect+0x123)[0x7fa13252c5c3]
        /opt/puppetlabs/puppet/lib/ruby/gems/2.5.0/gems/rugged-0.27.7/lib/rugged/rugged.so(+0x1e27c)[0x7fa1297b027c]
        /opt/puppetlabs/server/apps/r10k/lib/libgit2.so.27(+0xb3f83)[0x7fa129547f83]
        /opt/puppetlabs/server/apps/r10k/lib/libgit2.so.27(+0xb4bdf)[0x7fa129548bdf]
        /opt/puppetlabs/server/apps/r10k/lib/libgit2.so.27(+0xb0dcd)[0x7fa129544dcd]
        /opt/puppetlabs/server/apps/r10k/lib/libgit2.so.27(git_remote_connect+0x17d)[0x7fa12952013d]
        /opt/puppetlabs/server/apps/r10k/lib/libgit2.so.27(git_remote_fetch+0x8f)[0x7fa1295219df]
        /opt/puppetlabs/puppet/lib/ruby/gems/2.5.0/gems/rugged-0.27.7/lib/rugged/rugged.so(+0x1efb9)[0x7fa1297b0fb9]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1acef6)[0x7fa132642ef6]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc456)[0x7fa132652456]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bcef3)[0x7fa132652ef3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1c88)[0x7fa132647c88]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bd5ec)[0x7fa1326535ec]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_funcall_with_block+0x128)[0x7fa132655578]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_yield+0x108)[0x7fa132650288]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(rb_ary_each+0x3d)[0x7fa1324c697d]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1acef6)[0x7fa132642ef6]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc456)[0x7fa132652456]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bcef3)[0x7fa132652ef3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1a3b)[0x7fa132647a3b]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bbfc1)[0x7fa132651fc1]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc04b)[0x7fa13265204b]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc38c)[0x7fa13265238c]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc456)[0x7fa132652456]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bcef3)[0x7fa132652ef3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1c88)[0x7fa132647c88]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0xccb90)[0x7fa132562b90]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0xcd1c8)[0x7fa1325631c8]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0xcd323)[0x7fa132563323]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1acef6)[0x7fa132642ef6]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bc456)[0x7fa132652456]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1bcef3)[0x7fa132652ef3]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b1c88)[0x7fa132647c88]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x1b78ac)[0x7fa13264d8ac]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(+0x93ea5)[0x7fa132529ea5]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(ruby_exec_node+0x1d)[0x7fa13252bafd]
        /opt/puppetlabs/puppet/lib/libruby.so.2.5(ruby_run_node+0x1e)[0x7fa13252de7e]
        /opt/puppetlabs/puppet/bin/ruby[0x4008bb]
        /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa131791555]
        /opt/puppetlabs/puppet/bin/ruby[0x4008e9]
        ======= Memory map: ========
        00400000-00401000 r-xp 00000000 fd:06 1052365 /opt/puppetlabs/puppet/bin/ruby
        00600000-00601000 r--p 00000000 fd:06 1052365 /opt/puppetlabs/puppet/bin/ruby
        00601000-00602000 rw-p 00001000 fd:06 1052365 /opt/puppetlabs/puppet/bin/ruby
        02434000-0375d000 rw-p 00000000 00:00 0 [heap]
        7fa120000000-7fa120021000 rw-p 00000000 00:00 0
        7fa120021000-7fa124000000 ---p 00000000 00:00 0
        7fa126000000-7fa126015000 r-xp 00000000 fd:06 1184224 /opt/puppetlabs/puppet/lib/libgcc_s.so.1
        7fa126015000-7fa126214000 ---p 00015000 fd:06 1184224 /opt/puppetlabs/puppet/lib/libgcc_s.so.1
        7fa126214000-7fa126215000 r--p 00014000 fd:06 1184224 /opt/puppetlabs/puppet/lib/libgcc_s.so.1
        7fa126215000-7fa126216000 rw-p 00015000 fd:06 1184224 /opt/puppetlabs/puppet/lib/libgcc_s.so.1
        7fa126216000-7fa12622c000 r-xp 00000000 fd:02 403866 /usr/lib64/libresolv-2.17.so
        7fa12622c000-7fa12642c000 ---p 00016000 fd:02 403866 /usr/lib64/libresolv-2.17.so
        7fa12642c000-7fa12642d000 r--p 00016000 fd:02 403866 /usr/lib64/libresolv-2.17.so
        7fa12642d000-7fa12642e000 rw-p 00017000 fd:02 403866 /usr/lib64/libresolv-2.17.so
        7fa12642e000-7fa126430000 rw-p 00000000 00:00 0
        7fa126430000-7fa126436000 r-xp 00000000 fd:02 395999 /usr/lib64/libnss_dns-2.17.so
        7fa126436000-7fa126635000 ---p 00006000 fd:02 395999 /usr/lib64/libnss_dns-2.17.so
        7fa126635000-7fa126636000 r--p 00005000 fd:02 395999 /usr/lib64/libnss_dns-2.17.so
        7fa126636000-7fa126637000 rw-p 00006000 fd:02 395999 /usr/lib64/libnss_dns-2.17.so
        7fa126637000-7fa126666000 r-xp 00000000 fd:06 1185099 /opt/puppetlabs/puppet/lib/ruby/2.5.0/x86_64-linux/date_core.so
        7fa126666000-7fa126865000 ---p 0002f000 fd:06 1185099 /opt/puppetlabs/puppet/lib/ruby/2.5.0/x86_64-linux/date_core.so
        7fa126865000-7fa126866000 r--p 0002e000 fd:06 1185099 /opt/puppetlabs/puppet/lib/ruby/2.5.0/x86_64-linux/date_core.so
        7fa126866000-7fa126867000 rw-p 0002f000 fd:06 1185099 /opt/puppetlabs/puppet/lib/ruby/2.5.0/x86_64-linux/date_core.soAborted

Describe steps to reproduce…
Deploy server using customer's deployment process. Install PE2019.8.10, set up public/private keys for the control repo, setup code manager for deployments, run puppet code deploy --dry-run

Desired Behavior:
Desired behavior is the deployment completes.

Actual Behavior:
Deployment does not complete with ruby error stating corrupted size vs prev_size

Please take a moment and attach any relevant log output and/or manifests. This will help us immensely when troubleshooting the issue.

Examples:
Run puppet agent with --test --trace --debug

Relevant sections of /var/log/puppetlabs/puppetserver/puppetserver.log or any applicable logs from the same directory.

For more detailed information turn up the server logs by upping the log level in the server's logback.xml

Relevant sections of configurations files (puppet.conf, hiera.conf, Server's conf.d, defaults/sysconfig)

For memory issues with server heap dumps are also helpful.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

Geoffrey Rodgers (Jira)

unread,
Mar 9, 2022, 9:23:03 AM3/9/22
to puppe...@googlegroups.com
Geoffrey Rodgers commented on Bug PUP-11476
 
Re: Puppet code deploys fail with 2019.8.10 on RHEL 7.9

Note that the same error occurs when attempting to directly deploy an environment, e.g. 

puppet code deploy production --wait

 

results in the same error.

Reply all
Reply to author
Forward
0 new messages