Freeswitch reports Erlang node connection failure without any obvious reasons
635 views
Skip to first unread message
Yannick Guay
Jun 17, 2016, 3:49:26 PM6/17/16
to 2600hz-users
Good afternoon 2600hz-users,
I've just setup a brand new cluster for staging purposes and despite I cannot find any evidence of anything being wrong, Freeswitch constantly reports a connection failure warning. See below
[WARNING] mod_kazoo.c:752 Erlang node connection failed - ensure your cookie matches '[...]' and you are using a good nodename
Below are software version for all components that I use (it is a fresh setup):
- kazoo-freeswitch-R15B-1.4.23-2.el6.x86_64
- kazoo-configs-3.22-16.el6.noarch
- kazoo-ui-3.22-0.el6.noarch
- kazoo-R15B-3.22-57.el6.x86_64
- kazoo-bigcouch-R15B-0.4.x-1.el6.x86_64
- kazoo-kamailio-4.3.4-8.el6.x86_64
- kazoo-json-c-master-0.el6.x86_64
- kazoo-librabbitmq-master-1.el6.x86_64
I've check all nodes to make sure they all share the same cookie (which they do), I've issued sup -n ecallmgr ecallmgr_config get fs_nodes and sup -n ecallmgr ecallmgr_maintenance list_fs_nodes andI can see my unique fs_node gets listed as well. Also,RabbitMQ shows there are 4 active connections, 2 from Kamailio, 2 from eCallMgr and whApps. I'm even able to route calls through the cluster. Connection to BigCouch seems OK too. According to epmd, the node names also seem fine.
Am I missing anything? I honestly ran out of ideas for things to verify... Thanks for your help!
Best regards,
--Yannick
Juan
Jun 19, 2016, 7:25:24 PM6/19/16
to 2600hz-users
I had this problem when I didn't reboot after setting the hostname properly.
Make sure all your nodes can hostname -f. If you didn't reboot after setting the hostnames, it could be an issue (as freeswitch does not pick it up dynamically it seems).
Yannick Guay
Jun 20, 2016, 1:19:05 PM6/20/16
to 2600hz-users
Hi Juan,
Thanks for your reply. Unfortunately that doesn't seem to fix the issue.
The odd part is that it reports sending messages to ecallmgr and then a failure on the very next line in the logs:
2016-06-20 10:46:45.903144 [DEBUG] kazoo_node.c:1100 Sent erlang message to ecal...@apps1.domain.com <3.1471.0>
2016-06-20 10:46:46.083138 [WARNING] mod_kazoo.c:752 Erlang node connection failed - ensure your cookie matches '[]' and you are using a good nodename
Any suggestions? Anybody?
Regards,
-Yannick
James Aimonetti
Jun 21, 2016, 4:49:16 PM6/21/16
to 2600hz...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
First, check what FreeSWITCH/mod_kazoo is presenting:
fs_cli -x 'erlang status'
Listening for new Erlang connections on 0.0.0.0:8031 with cookie change_me
Registered as Erlang node freeswitch@{HOSTNAME}
...
The cookie should match what the /etc/kazoo/config.ini has for cookie in
the [ecallmgr] section.
The node name, freeswitch@{HOSTNAME}, should match what you used to tell
ecallmgr about FreeSWITCH.
I'm pretty confident your cookie value in FreeSWITCH/mod_kazoo is borked
though. Check /etc/kazoo/freeswitch/autoload_configs/kazoo.conf.xml to
see what the cookie is set to, make it match ecallmgr's cookie, and
restart mod_kazoo.
Yannick Guay writes:
> Hi Juan,
>
> Thanks for your reply. Unfortunately that doesn't seem to fix the issue.
>
> The odd part is that it reports sending messages to ecallmgr and then a
> failure on the very next line in the logs:
> *2016-06-20 10:46:45.903144 [DEBUG] kazoo_node.c:1100 Sent erlang message
> to ecal...@apps1.domain.com <3.1471.0>*
> *2016-06-20 10:46:46.083138 [WARNING] mod_kazoo.c:752 Erlang node
>>> - kazoo-configs-3.22-16.el6.noarch
>>> - kazoo-ui-3.22-0.el6.noarch
>>> - kazoo-R15B-3.22-57.el6.x86_64
>>> - kazoo-bigcouch-R15B-0.4.x-1.el6.x86_64
>>> - kazoo-kamailio-4.3.4-8.el6.x86_64
>>> - kazoo-json-c-master-0.el6.x86_64
>>> - kazoo-librabbitmq-master-1.el6.x86_64
>>> -n ecallmgr ecallmgr_maintenance list_fs_nodes* andI can see my unique
James Aimonetti
Lead Systems Architect
"If Dialyzer don't care, I don't care"
2600HzPDX | http://2600hz.com
sip:ja...@2600hz.com
tel:415.886.7905
irc:mc_ @ freenode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJXaahKAAoJENTKa+JPXCVgHeYIALaEK9xMKX6JDaJFCJ/63Udq
wB1kCxMqrUwNIRfvJOSI1OnqKXp3V0xnyptvbQvI+YBpLR/6Bf2pRgGEUJy6LWHJ
Nb1iAVKhcHRJK0ob5+3K+M/kY+YPll1xewb43laS2b2/nhiJAkktPh2o6IU5QcK0
LIKdsfUKaNfLnVV9neXwT05cGShN3IgdHQ7qq640Mz8t7RTj3ltdytVGue5qktan
45s+y72XfMSc9DlOhuo0mzMcUkNzoUeUuRHW9wj96HTqSm+pZiftYNkbuKBChQoW
L2WH8sThdFfsboqjJhRJtkY1UNsB6LcoR7M1BT8rX3icz+zJnqXtyuKgSejBBQ8=
=aKMR
-----END PGP SIGNATURE-----
Hash: SHA256
First, check what FreeSWITCH/mod_kazoo is presenting:
fs_cli -x 'erlang status'
Listening for new Erlang connections on 0.0.0.0:8031 with cookie change_me
Registered as Erlang node freeswitch@{HOSTNAME}
...
The cookie should match what the /etc/kazoo/config.ini has for cookie in
the [ecallmgr] section.
The node name, freeswitch@{HOSTNAME}, should match what you used to tell
ecallmgr about FreeSWITCH.
I'm pretty confident your cookie value in FreeSWITCH/mod_kazoo is borked
though. Check /etc/kazoo/freeswitch/autoload_configs/kazoo.conf.xml to
see what the cookie is set to, make it match ecallmgr's cookie, and
restart mod_kazoo.
Yannick Guay writes:
> Hi Juan,
>
> Thanks for your reply. Unfortunately that doesn't seem to fix the issue.
>
> The odd part is that it reports sending messages to ecallmgr and then a
> failure on the very next line in the logs:
> to ecal...@apps1.domain.com <3.1471.0>*
> *2016-06-20 10:46:46.083138 [WARNING] mod_kazoo.c:752 Erlang node
> connection failed - ensure your cookie matches '[]' and you are using a
> good nodename*
>
> Any suggestions? Anybody?
>
> Regards,
> -Yannick
>
>
> Le dimanche 19 juin 2016 19:25:24 UTC-4, Juan a écrit :
>>
>> I had this problem when I didn't reboot after setting the hostname
>> properly.
>> Make sure all your nodes can hostname -f. If you didn't reboot after
>> setting the hostnames, it could be an issue (as freeswitch does not pick it
>> up dynamically it seems).
>>
>> On Saturday, June 18, 2016 at 7:49:26 AM UTC+12, Yannick Guay wrote:
>>>
>>> Good afternoon 2600hz-users,
>>>
>>> I've just setup a brand new cluster for staging purposes and despite I
>>> cannot find any evidence of anything being wrong, Freeswitch constantly
>>> reports a connection failure warning. See below
>>>
>>> [WARNING] mod_kazoo.c:752 Erlang node connection failed - ensure your
>>> cookie matches '[...]' and you are using a good nodename
>>>
>>> Below are software version for all components that I use (it is a fresh
>>> setup):
>>>
>>> - kazoo-freeswitch-R15B-1.4.23-2.el6.x86_64
> Any suggestions? Anybody?
>
> Regards,
> -Yannick
>
>
> Le dimanche 19 juin 2016 19:25:24 UTC-4, Juan a écrit :
>>
>> I had this problem when I didn't reboot after setting the hostname
>> properly.
>> Make sure all your nodes can hostname -f. If you didn't reboot after
>> setting the hostnames, it could be an issue (as freeswitch does not pick it
>> up dynamically it seems).
>>
>> On Saturday, June 18, 2016 at 7:49:26 AM UTC+12, Yannick Guay wrote:
>>>
>>> Good afternoon 2600hz-users,
>>>
>>> I've just setup a brand new cluster for staging purposes and despite I
>>> cannot find any evidence of anything being wrong, Freeswitch constantly
>>> reports a connection failure warning. See below
>>>
>>> [WARNING] mod_kazoo.c:752 Erlang node connection failed - ensure your
>>> cookie matches '[...]' and you are using a good nodename
>>>
>>> Below are software version for all components that I use (it is a fresh
>>> setup):
>>>
>>> - kazoo-configs-3.22-16.el6.noarch
>>> - kazoo-ui-3.22-0.el6.noarch
>>> - kazoo-R15B-3.22-57.el6.x86_64
>>> - kazoo-bigcouch-R15B-0.4.x-1.el6.x86_64
>>> - kazoo-kamailio-4.3.4-8.el6.x86_64
>>> - kazoo-json-c-master-0.el6.x86_64
>>> - kazoo-librabbitmq-master-1.el6.x86_64
>>>
>>> I've check all nodes to make sure they all share the same cookie (which
>>> they do), I've issued *sup -n ecallmgr ecallmgr_config get fs_nodes* and *sup
>>> I've check all nodes to make sure they all share the same cookie (which
>>> -n ecallmgr ecallmgr_maintenance list_fs_nodes* andI can see my unique
>>> fs_node gets listed as well. Also,RabbitMQ shows there are 4 active
>>> connections, 2 from Kamailio, 2 from eCallMgr and whApps. I'm even able to
>>> route calls through the cluster. Connection to BigCouch seems OK too.
>>> According to epmd, the node names also seem fine.
>>>
>>> Am I missing anything? I honestly ran out of ideas for things to
>>> verify... Thanks for your help!
>>>
>>> Best regards,
>>> --Yannick
>>>
>>
- --
>>> connections, 2 from Kamailio, 2 from eCallMgr and whApps. I'm even able to
>>> route calls through the cluster. Connection to BigCouch seems OK too.
>>> According to epmd, the node names also seem fine.
>>>
>>> Am I missing anything? I honestly ran out of ideas for things to
>>> verify... Thanks for your help!
>>>
>>> Best regards,
>>> --Yannick
>>>
>>
James Aimonetti
Lead Systems Architect
"If Dialyzer don't care, I don't care"
2600HzPDX | http://2600hz.com
sip:ja...@2600hz.com
tel:415.886.7905
irc:mc_ @ freenode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJXaahKAAoJENTKa+JPXCVgHeYIALaEK9xMKX6JDaJFCJ/63Udq
wB1kCxMqrUwNIRfvJOSI1OnqKXp3V0xnyptvbQvI+YBpLR/6Bf2pRgGEUJy6LWHJ
Nb1iAVKhcHRJK0ob5+3K+M/kY+YPll1xewb43laS2b2/nhiJAkktPh2o6IU5QcK0
LIKdsfUKaNfLnVV9neXwT05cGShN3IgdHQ7qq640Mz8t7RTj3ltdytVGue5qktan
45s+y72XfMSc9DlOhuo0mzMcUkNzoUeUuRHW9wj96HTqSm+pZiftYNkbuKBChQoW
L2WH8sThdFfsboqjJhRJtkY1UNsB6LcoR7M1BT8rX3icz+zJnqXtyuKgSejBBQ8=
=aKMR
-----END PGP SIGNATURE-----
Sergey Safarov
Jun 21, 2016, 5:26:22 PM6/21/16
to 2600hz-users
Probably FreeSwitch node longname in FreeSwitch setting different from FreeSwitch node longname in Kazoo database.
Sergey
понедельник, 20 июня 2016 г., 2:25:24 UTC+3 пользователь Juan написал:
понедельник, 20 июня 2016 г., 2:25:24 UTC+3 пользователь Juan написал:
Yannick Guay
Jun 23, 2016, 9:53:37 AM6/23/16
to 2600hz-users
Hi James, Sergey,
It seems these warnings do not appear in FreeSWITCH logs anymore, and I haven't changed anything specific to solve this.
Thank to both of you for your assistance, really appreciate it!
-Yannick
Reply all
Reply to author
Forward
0 new messages