API call to certificate_request deletes the request

118 views
Skip to first unread message

Mikhail Simin

unread,
Jun 28, 2015, 1:49:21 PM6/28/15
to puppet...@googlegroups.com

I'm using Puppet 3.7.3 and I observe this strange behavior when using the API to sign a certificate:


==> /var/log/apache.log <==
Jun 28 17:18:07.000000 prod-puppetca apache: 127.0.0.1 prod-puppetca:8140 - - [28/Jun/2015:17:18:03 +0000] "PUT /production/certificate_request/prod-clientbox HTTP/1.1" 200 1582 "-" "python-requests/2.7.0 CPython/2.7.6 Linux/3.13.0-46-generic"

==> /var/log/daemon.log <==
Jun 28 17:18:03.000000 prod-puppetca puppet-master[27451]: prod-clientbox has a waiting certificate request
Jun 28 17:18:07.000000 prod-puppetca puppet-master[27451]: Signed certificate request for prod-clientbox
Jun 28 17:18:07.000000 prod-puppetca puppet-master[27451]: Removing file Puppet::SSL::CertificateRequest prod-clientbox at '/var/lib/puppet/ssl/ca/requests/prod-clientbox.pem'


For some reason a single PUT call to `certificate_request/` signs the CSR and then also removes it!


Under normal circumstances (when the CSR does not get removed) I have a follow up API call for `certificate_status/` with {"desired_state":"signed"} passed in. However when the CSR is removed, this no longer works because puppet refuses with the following message: 


Cannot sign for host prod-clientbox without a certificate request


Why does the CSR get removed with the same API call that uploads it?

adaryl.l....@gmail.com

unread,
Jun 28, 2015, 6:16:18 PM6/28/15
to puppet...@googlegroups.com
When I go to start my puppet agent using:
sudo /opt/puppetlabs/bin/puppet resource service puppet ensure=running enable=true.
 
I get:
 
sh: 0: getcwd() failed: No such file or directory
 
When I try again, I discover that the /opt directory has magically disappeared. Any ideas?
 
B.

Daniel Dreier

unread,
Jun 28, 2015, 6:52:15 PM6/28/15
to puppet...@googlegroups.com
I'd have an easier time trying to reproduce the problem if you include information on what platform you're running on, what version of puppet you installed, and any other setup steps you may have taken. Ideally, if you can reproduce this consistently via a shell script in a vagrant environment it'll be much easier to troubleshoot and you'll get faster feedback.



--
Daniel Dreier
Technical Operations Engineer
GPG: BA4379FD

adaryl.l....@gmail.com

unread,
Jun 28, 2015, 7:01:02 PM6/28/15
to puppet...@googlegroups.com
Unfortunately I’m not sophisticated enough to use Vagrant top of which I’m not installing in a VM. I’m using a cluster of 3 bare metal machines running Ubuntu 14.4. I’ve followed every step prescribed to the letter in this link: http://docs.puppetlabs.com/puppet/4.2/reference/install_linux.html
 
What other information can I pass along?
 
 
Adaryl "Bob" Wakefield, MBA
Principal
Mass Street Analytics, LLC
913.938.6685
www.linkedin.com/in/bobwakefieldmba
Twitter: @BobLovesData
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAGk8suY5XZ7b1dVf2Ap5Xv9%3DhDBgoQo2%2B%2BACRjqec-QjD00_Lg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Daniel Dreier

unread,
Jun 28, 2015, 7:30:03 PM6/28/15
to puppet...@googlegroups.com
Here are the steps I've taken to try and reproduce the issue you're describing:

vagrant init ubuntu/trusty64
vagrant ssh
sudo dpkg -i puppetlabs-release-pc1-trusty.deb
sudo apt-get update
sudo apt-get install puppet-agent -y
sudo /opt/puppetlabs/bin/puppet resource service puppet ensure=running enable=true

When I run it, I get the following output:

Notice: /Service[puppet]/ensure: ensure changed 'stopped' to 'running'
service { 'puppet':
  ensure => 'running',
  enable => 'true',
}

I'd encourage you to try out vagrant to simplify troubleshooting -- it really is just a matter of installing virtualbox, vagrant, and running "vagrant init ubuntu/trusty64", "vagrant up" and "vagrant ssh" and you're in a VM. It'll save hours compared to reprovisioning bare metal hardware.

Can you suggest any major configuration differences we might have between the plain-vanilla ubuntu 14.04 vagrant environment and what you're running?


For more options, visit https://groups.google.com/d/optout.

adaryl.l....@gmail.com

unread,
Jun 28, 2015, 7:46:28 PM6/28/15
to puppet...@googlegroups.com
When I run your commands, I get the same out put. There are a few difference between what you wrote and my initial work.
1. sudo apt-get install puppet-agent –y <—I didn’t use the –y
2. It asked me to install puppet-common which I did.
3. I changed the configuration settings in /etc/puppet/puppet.conf so that, in the main section, I have a line server=[master hostname]
4. I went back through my command history and discovered that I tried to apt-get install before I apt-get update. I had to go back and update then I apt-get install again.
5. I’m logged in as root and I don’t use sudo in my commands.
 
Those are the only differences.

adaryl.l....@gmail.com

unread,
Jun 28, 2015, 8:02:08 PM6/28/15
to puppet...@googlegroups.com
So everything seems to be running on my install attempt. One of the last instructions is to log into the master and see if there are any outstanding request. When I run: sudo /opt/puppetlabs/bin/puppet cert list, I get no results.

Daniel Dreier

unread,
Jun 28, 2015, 8:53:13 PM6/28/15
to puppet...@googlegroups.com
I hope this doesn't sound pedantic, but when you say you get the same output do you mean you got the same output as I did, or the same error output as you originally posted about?

Also, could you please run "apt --installed list | grep puppet" and copy-paste the output, just to confirm which puppet packages you ended up with?

That said, I'm stumped. The only advice I can give is to start from a known, clean environment, and script 100% of the setup needed to reproduce the problem, in order to make it feasible for somebody on this list to see the problem first-hand. That may be as easy as installing vagrant and converting part of your bash history from the old box into a setup shell script to run on the vagrant environment.


For more options, visit https://groups.google.com/d/optout.

Josh Cooper

unread,
Jun 29, 2015, 2:16:24 AM6/29/15
to puppet...@googlegroups.com
It sounds like you have autosign[1] enabled. Check /etc/puppet/puppet.conf or in the script that starts your CA.

Josh

Adaryl Wakefield

unread,
Jun 29, 2015, 3:45:32 AM6/29/15
to puppet...@googlegroups.com
Grep output:
puppet-agent/trusty,now 1.2.1-1trusty amd64 [installed]
puppetlabs-release-pc1/trusty,now 0.9.2-1trusty all [installed]

I got the same output you did. After checking the services, they appear to be up and running. The issue I'm having now and that I posted about in another thread is that I'm supposed to run:
sudo /opt/puppetlabs/bin/puppet cert list to sign the certificates, but when I run that code, I get nothing back indicating that there are no outstanding cert request. From that, it would seem to suggest that the agents aren't contacting the master.

B.

Mikhail Simin

unread,
Jul 6, 2015, 1:32:51 PM7/6/15
to puppet...@googlegroups.com
Thanks Josh, you hit the nail on the head.

Disabling autosign makes my API calls work as expected. But why does autosign delete the CSR? The docs don't say anything about this. Can I disable that feature somehow?
I need autosigning to be enabled for other purposes, and also be able to invoke API calls as I do right now.

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/LCAuO4Wo_d8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5acc5158-2740-4167-9404-4651ed728bc7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Mikhail Simin, Ph.D
The Private Social Network for Neighborhoods

Mikhail Simin

unread,
Jul 6, 2015, 1:57:47 PM7/6/15
to puppet-users
Found the explanation in the actual code, rather than the docs. It's an interesting security implementation. I'll have to figure out how to work with it.

Thank you!
Mikhail
Reply all
Reply to author
Forward
0 new messages