Puppet web server stops after being started

[Michael Wildey]

I installed an puppetserver just like ist is written in the manual https://docs.puppet.com/puppet/4.5/reference/index.html on Debian Jessie.

My Problem is that as soon as I want to get a cert for an agent it says that the puppetserver doesn't listen on Port 8140.

Now I found out that the web server stops immediately after being started, but I don't know why.

Any help is welcome.

[Rob Nelson]

You can check the logs of the puppetserver startup. A common reason is that VMs have less RAM than the process expects to be allocated. You can either increase the RAM or adjust the puppetserver settings to use less RAM, with impact to the performance of course. See https://docs.puppet.com/puppetserver/latest/install_from_packages.html#system-requirements if that seems a likely cause.

Rob Nelson
rnel...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/93e0e0e8-2b10-48ea-82c3-40fa8ec86d91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Wildey]

I already set the RAM on 2 GB. That was the first thing I though. But still it shuts down the web server in less than a second.

The Log said it is Initialzing, starting and shutting down the web server in 0,2 seconds. This is the only information I get from the puppetserver.log. Is there a other log I don't know by now where I can get more information about it?

[Matthaus Owens]

Michael,

One good way of troubleshooting this sort of thing is to use the foreground command. It will attempt to start the process but log everything to the console so it's easy to find. `/opt/puppetlabs/bin/puppetserver foreground` should do the right thing. If you can paste or gist the log output that would help a lot in figuring out what is going on.

-Matthaus

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/e2741ca6-5d04-4949-b214-4b11775c4d37%40googlegroups.com.

[Michael Wildey]

Ok, I did it now in foreground, just like you told me. 

What I have done before is that I changed the webserver.conf just like it is described here >>https://docs.puppet.com/puppetserver/latest/external_ca_configuration.html#web-server-configuration<<

The output is attached.

Thanks for your hep

Michael

[Matthaus Owens]

Michael,

The important bit of that log is the following:

2016-08-03 17:12:31,742 ERROR [async-dispatch-2] [p.t.internal] Error during service init!!!
java.lang.IllegalArgumentException: Missing authorization service configuration.

That indicates that  /etc/puppetlabs/puppetserver/conf.d/auth.conf (https://docs.puppet.com/puppetserver/2.4/config_file_auth.html) is empty or doesn't exist. The file that is laid down by default by our packages is here: https://github.com/puppetlabs/puppetserver/blob/master/ezbake/config/conf.d/auth.conf. If you drop that file into  /etc/puppetlabs/puppetserver/conf.d/auth.conf the service should start successfully. If not, attach the new log and I'd be happy to help debug further.

Hope that helps,

-Matthaus

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/695e4bc4-cc4f-4c96-98ef-5fa06705a53a%40googlegroups.com.

[Michael Wildey]

Matthaus,

I found that I renamed the auth.conf by mistake. Now I renamed it again and then I started the hole server again.

After that I tried to start the server again in foreground. The output is attached.

Now there is written "Finished TK main lifecycle, shutting down Clojure agent threads" 

I am getting confused. I've downloaded a bundle from puppet and now it wont work. I can't imagine what went wrong during the installation and I blive I am not the first one to install puppetserver on a VM. Especially after i already installed   it on an other VM.

Hope you can help me her

Michael

[Matthaus Owens]

Michael,

If you don't mind posting the contents of your auth.conf as well, that could help in figuring out what is going on. It looks like there is a problem with the contents of the file, but without seeing the file, it is hard to know what specifically is wrong.

Thanks,

Matthaus

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/524f9812-0df8-43d0-8fa7-773cd28e4e42%40googlegroups.com.

[Jeremy Barlow]

Michael,

Did you happen to move the file installed by default at /etc/puppetlabs/puppet/auth.conf to /etc/puppetlabs/puppetserver/conf.d/auth.conf.  The formats for these two auth.conf files are completely different and cannot be used interchangeably.

The file at /etc/puppetlabs/puppet/auth.conf has a "custom" format which looks something like this:

path /puppet/v3/environments
method find
allow *

...

The file at /etc/puppetlabs/puppetserver/conf.d/auth.conf should be in the HOCON format and should look something like this:

authorization: {
    version: 1
    rules: [
        {
            # Allow nodes to retrieve their own catalog
            match-request: {
                path: "^/puppet/v3/catalog/([^/]+)$"
                type: regex
                method: [get, post]
            }
            allow: "$1"
            sort-order: 500
            name: "puppetlabs catalog"
        },
...
  ]
}

For reference, you can see the complete content of the Puppet Server auth.conf file for the 2.4.0 packages here.

--- Jeremy

[Michael Wildey]

Jeremy,

it looks like the /etc/puppetlabs/puppetserver/conf.d/auth.conf is gone and has been replaced by the /etc/puppetlabs/puppet/auth.conf

There is no differnence between both.

I am now going to replace it with the auth.conf file wich is in the link you've being posting. I hope this will help.

After I did that I restarted the server and it is still shutting down the web server. I am going to attach the output of the log file puppetserver.log and what was written in foreground during the start

Michael

[Jeremy Barlow]

Michael,

It looks like it got further that time but a different error came up this time:

java.lang.IllegalStateException: Cannot initialize master with partial state; need all files or none.
Found:
/etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem
Missing:
/etc/puppetlabs/puppet/ssl/certs/puppetserver.localdomain.pem

This error, unfortunately, sounds like a bug previously reported in the Puppet JIRA tracker as SERVER-528.  The simplest way to work around this problem would be to just run...

rm -rf /etc/puppetlabs/puppet/ssl

... and restart the puppetserver service.  puppetserver should recreate the "ssl" directory after it restarts and re-generate its own private key and certificate.

Hope it starts up for you after that.

--- Jeremy

[Michael Wildey]

Jeremy,

I removed the ssl folder and restarted the service.

But now it say that there is one file missing. 

>>java.lang.IllegalStateException: Cannot initialize master with partial state; need all files or none.

Found:

/etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem

Missing:

/etc/puppetlabs/puppet/ssl/certs/puppetserver.localdomain.pem<<

Is it possible to delete >>/etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem<<?

I attached the output of the log. Just in case this wasn't the problem.

Michael

[Jeremy Barlow]

On Monday, August 29, 2016 at 7:44:02 AM UTC-7, Michael Wildey wrote:

 

I removed the ssl folder and restarted the service.

But now it say that there is one file missing. 

>>java.lang.IllegalStateException: Cannot initialize master with partial state; need all files or none.

Found:

/etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem

Missing:

/etc/puppetlabs/puppet/ssl/certs/puppetserver.localdomain.pem<<

Is it possible to delete >>/etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem<<?

I attached the output of the log. Just in case this wasn't the problem.

It sounds like you might have been able to remove the ssl directory but that you might have some process running - like a cron job running the puppet agent - which may be recreating the private key before you are able to restart the puppetserver service successfully.  Maybe an easier approach would be to just generate a certificate for your master from the command line so that both the private key and certificate are in place when the puppetserver service starts.  You could try running the following command:

/opt/puppetlabs/puppet/bin/puppet cert generate puppetserver.localdomain

After doing that, try to restart the puppetserver service again and see if it starts this time.

Very sorry that this has been so painful of an issue to workaround.  I really hope we're able to implement a fix for SERVER-528 sometime soon.

--- Jeremy

[Michael Wildey]

It looks to me like fixing a problem and causing an other problem 

Now it says on the log file that there is still a problem. It is 

>>unable to open 'ssl-cert' file /path/to/master.pem<<

and after that it is shuting down the web server again

The output of the log is attached to this post 

Just an idea, could it be better to remove the installation completly and try to reinstall everything?

[Jeremy Barlow]

It sounds like you may have defined a setting called 'ssl-cert' with a literal value of '/path/to/master.pem' in your /etc/puppetlabs/puppetserver/conf.d/webserver.conf file.  Maybe this was done following the instructions for "External CA Configuration" from this page?

If you don't intend to use an External CA configuration, you could omit the 'ssl-cert', 'ssl-key', 'ssl-ca-cert', 'ssl-cert-chain', and 'ssl-crl-path' settings from the webserver.conf file, as is done in the default file installed by the Puppet Server package.  If you do intend for these settings to be defined, note that the values you insert into them would need to point to the actual locations where the different files live.  '/path/to/...' is just a placeholder.  Using your example from earlier, the "real" locations of those files could be:

webserver: {
  ...
  ssl-cert: /etc/puppetlabs/puppet/ssl/certs/puppetserver.localdomain.pem
  ssl-key: /etc/puppetlabs/puppet/ssl/private_keys/puppetserver.localdomain.pem
  ssl-ca-cert: /etc/puppetlabs/puppet/ssl/certs/ca.pem
  ...
}

Given all of the issues that you have run into so far, though, I agree that it might be better to just remove / reinstall the package completely.