Port 8140/TCP (PuppetServer) close

133 views
Skip to first unread message

Aécio

unread,
Mar 11, 2018, 7:59:05 AM3/11/18
to
Hello guys!

I have the following problem: every 30 or 40 minutes, I get a notification that port 8140 / TCP (from PuppetServer) is closed and puppet-agent (from client servers) can not get the catalog and / or send reports.

Looking at the log in /var/log/puppetlabs/puppetserver/puppetserver.log and watching the process, I saw that the puppetserver service did not stop.

Can anyone help me solve this problem?

What have I done?


2) I installed PuppetServer 2.8.1 (compatible with Puppet4.x) on a server with 6GB of RAM and 2 vCPU and changed the following settings:

# /etc/default/puppetserver
JAVA_ARGS = "- Xms2G -Xmx2G"

# /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf
max-active-instances: 4

Unfortunately, the issue has not been resolved.

Thanks for the attention and any help.

Aécio Pires

Rob Nelson

unread,
Mar 11, 2018, 8:37:56 AM3/11/18
to puppe...@googlegroups.com
You should probably start by verifying it is listening on port 8140, then look at any firewalls between the agents and the master - including iptables or other OS-level firewalls - to ensure they are allowing the traffic. 

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CALCxXJhp5806ZZPS9T0KNBUfhcpbeAmEK3C1s3RR8OUOAT9oFw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
Rob Nelson

Aécio

unread,
Mar 11, 2018, 9:49:10 AM3/11/18
to puppe...@googlegroups.com
Hello, Rob!

Thanks for attention.

There is no firewall between PuppetServer and Agent.

As I said before, the PuppetServer port is open and has communication between them, but every 30 or 40 minutes the PuppetServer port is closed for 60 seconds and then it stays open again.

The strange thing is that this occurs several times throughout the day. And in the logs it does not show that the PuppetServer service has been restarted

Aécio Pires
Book of Puppet => novatec.com.br/livros/puppet
Book of Zabbix => novatec.com.br/livros/zabbix

2018-03-11 9:37 GMT-03:00 Rob Nelson <rnel...@gmail.com>:
You should probably start by verifying it is listening on port 8140, then look at any firewalls between the agents and the master - including iptables or other OS-level firewalls - to ensure they are allowing the traffic. 

On Sun, Mar 11, 2018 at 7:59 AM Aécio <aecio...@gmail.com> wrote:
Hello guys!

I have the following problem: every 30 or 40 minutes, I get a notification that port 8140 / TCP (from PuppetServer) is closed and puppet-agent (from client servers) can not get the catalog and / or send reports.

Looking at the log in /var/log/puppetlabs/puppetserver/puppetserver.log and watching the process, I saw that the puppetserver service did not stop.

Can anyone help me solve this problem?

What have I done?


2) I installed PuppetServer 2.8.1 (compatible with Puppet4.x) on a server with 6GB of RAM and 2 vCPU and changed the following settings:

# /etc/default/puppetserver
JAVA_ARGS = "- Xms2G -Xmx2G"

# /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf
max-active-instances: 4

Unfortunately, the issue has not been resolved.

Thanks for the attention and any help.

Aécio Pires

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
--
Rob Nelson

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAC76iT9mmPvWdPHJt6yR31pFiujuVwpvkX_NvgeR7-%3Ddv1-KNQ%40mail.gmail.com.

Justin Stoller

unread,
Mar 12, 2018, 5:05:21 PM3/12/18
to puppe...@googlegroups.com
On Sun, Mar 11, 2018 at 4:58 AM, Aécio <aecio...@gmail.com> wrote:
Hello guys!

I have the following problem: every 30 or 40 minutes, I get a notification that port 8140 / TCP (from PuppetServer) is closed and puppet-agent (from client servers) can not get the catalog and / or send reports.

Looking at the log in /var/log/puppetlabs/puppetserver/puppetserver.log and watching the process, I saw that the puppetserver service did not stop.

Can anyone help me solve this problem?

What have I done?


2) I installed PuppetServer 2.8.1 (compatible with Puppet4.x) on a server with 6GB of RAM and 2 vCPU and changed the following settings:

# /etc/default/puppetserver
JAVA_ARGS = "- Xms2G -Xmx2G"

Is there a reason not to give it more ram? I see you have 6gs on the system, usually folks put the server on a dedicated box and give it all but 1 or 2 gigs of the memory available, having said that for a small installation 2 gigs should be enough.

# /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf
max-active-instances: 4

Usually we default max-active-instances to (number of cps - 1) with a max of 4. That would give you a max-active-instances of 1. 4 is too many for a box with 2 vCPUs and 2gs of memory.

This is my own recency bias but we were just working on a problem where all JRuby instances were flushed at the same time and caused an unacceptable pause. The port shouldn't close during that time however, requests should just pile up. If it was a JRuby instance issue, knowing the number of agents checking in and tuning max-requests-per-instance might help (in addition to lowering the number of max-active-instances), you would also see all of the JRuby instances being flushed and new ones primed in the logs during the downtime.

Do you have anything HUPing the server at these times? Reloading the server shouldn't take nearly that long but might take an extended period with those memory and instance allocations.
You wouldn't see the whole process go down, but you would see some shutdown startup functions in the logs, and at debug see an actual notice that it was happening because of a HUP. 
  

Hope that helps,
Justin

Unfortunately, the issue has not been resolved.

Thanks for the attention and any help.

Aécio Pires

--

Aécio

unread,
Mar 13, 2018, 3:34:14 PM3/13/18
to puppe...@googlegroups.com
Hello Justin!

Thank you very much for your attention and for the tips.

I set the PuppetServer memory from 2 to 4 GB.

# /etc/default/puppetserver
JAVA_ARGS = "- Xms4G -Xmx4G"

I also adjusted the number of JRuby instances from 4 to 2.

# /etc/puppetlabs/puppetserver/conf.d/puppetserver.conf
max-active-instances: 2


It's been 4 hours since I made this configuration and the problem was solved.

It used to happen every 30 or 40 minutes.


Abraço e fica com Deus.

Aécio Pires
Livro de Puppet => novatec.com.br/livros/puppet
Livro de Zabbix => novatec.com.br/livros/zabbix

Reply all
Reply to author
Forward
0 new messages