408, 480 Call Failed

[Roger Northrop]

I am getting 408, and 480 errors on my 15.08 version SipxCom.

Note this is how I have this set up:

SBC Acme Packet Port 5060 UDP > SipXcom Port 5060 TCP

G711

NOTE: I am NAT'ing in the Acme Packet to the SipXcom

408 Request Timeout

Couldn't find the user in time. The server could not produce a response within a suitable amount of time, for example, if it could not determine the location of the user in time. The client MAY repeat the request without modifications at any later time. 

480 Temporarily Unavailable

Callee currently unavailable

I am taking inbound calls to Voicemail, and it seems that I can get 4 - 6 calls with in a couple of seconds.

I have 4 Gig of RAM and Quad Core 2.9Ghz.  I am upgrading RAM to 8 Gig hoping that will help.

Also do I need to up the Maximum Concurrent Threads in the Sip Proxy settings, will this help?

I have some users saying that they call and get a fast busy, but then call right back and get in...

What do I need to do to make sure that I can get 100% Completion rate on inbound calls

 

[Todd Hodgen]

Are you making an assumption that the call is being blocked by the sipXcom system, or have you taken packet captures at its interface to see if that is the issue?  

 

I’d take a failed call capture and look to see what is actually going on.  Could be system, but more likely something else if we are only talking about a few calls.

 

Todd R. Hodgen

President / Founder

Sound IP Telecom

http://SIPTelecom.systems

206-432-4344 - Direct

206-390-4689 - Cell

 

Your Puget Sound Telcom Partner

 

Do you Like Snow, Rain, Sleet, Hail, Thunder, Lightning?

If you do, then you will love the Cloud!

 

We appreciate your business referrals!

A Division of Misiu Systems LLC

--
You received this message because you are subscribed to the Google Groups "sipxcom-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sipxcom-user...@googlegroups.com.
To post to this group, send email to sipxco...@googlegroups.com.
Visit this group at http://groups.google.com/group/sipxcom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/sipxcom-users/8b142daf-9e56-4434-a6e7-93653ffe4923%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joe Micciche]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/24/2015 04:25 PM, Roger Northrop wrote:
> I am getting 408, and 480 errors on my 15.08 version SipxCom.
>
> Note this is how I have this set up:
>
>
> SBC Acme Packet Port 5060 UDP > SipXcom Port 5060 TCP G711

Rotate your logs on the Acme, set to debug level on sipd and maybe
mbcd, then notify/tail the logfiles (or export them). Or better yet,
set up a capture receiver if you don't already have one.

That call load is nothing, so comparing the Acme logs with sipX proxy
logs and/or tcpdump will paint a full picture for you.

joe

>
> NOTE: I am NAT'ing in the Acme Packet to the SipXcom
>
>
> 408 Request Timeout Couldn't find the user in time. The server
> could not produce a response within a suitable amount of time, for
> example, if it could not determine the location of the user in
> time. The client MAY repeat the request without modifications at
> any later time.
>

> 480 Temporarily UnavailableCallee currently unavailable

>
> I am taking inbound calls to Voicemail, and it seems that I can get
> 4 - 6 calls with in a couple of seconds. I have 4 Gig of RAM and
> Quad Core 2.9Ghz. I am upgrading RAM to 8 Gig hoping that will
> help. Also do I need to up the Maximum Concurrent Threads in the
> Sip Proxy settings, will this help?
>
> I have some users saying that they call and get a fast busy, but

> then call right back and get in...What do I need to do to make sure

> that I can get 100% Completion rate on inbound calls
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlZU2TgACgkQJHjEUGX5D+Fs0ACgkzLxlaa4eTuZLEzcAqFUpgcm
I0kAnjP5iXGZIVBlaPubx/EeKJyCqsuM
=ACvD
-----END PGP SIGNATURE-----

[Roger Northrop]

Todd:

  this is a live system, and I would need to obfuscate all of the numbers / IP's, etc.. in order to give you the full trace... but here is a screen shot of my HOMR showing the progression of the Invite to the 482 and 480 codes.  "note" the 482 code = Loop, and this does not always take place, it just happened to be on this one.  but what you will notice is that the SIP Invite gets all the way to setting up to the IVR of the SipXecs but then errors.  See attached screen shot.

I am getting an average of 8 of these scattered with in 1750 Sip Sessions during the day.  it almost looks like this is happening at the SipXecs, and not in the network or in the Acme that I have running

[Roger Northrop]

Ok:

  I have been monitoring both my CDR log, and learned that this Alarm that I got was at the exact moment that i received a 480 Sip error, and the called failed.

this seems that this Alarm is directly related to my 480 and 408 errors.  

I have 8 Cores running at 2.9GHZ, and 8 Gig of Ram running on CentOS 6.7  64bit. with 15,000 rpm drives, CPU usage = 1%

sipregistrar[3328]: ALARM_MONGODB_SLOW_READ Last Mongo read took a long time: document: node.registrar delay: 255 milliseconds

I thought I got rid of the Slow Read errors.  What adjustments can I make to the SipProxy and DB Settings to rid my system of this since I do not want any calls to fail.

[Michael Picher]

You might want to make NUMA is disabled in the BIOS...  has known issues with Mongo.

Michael Picher, VP of Product Innovation
eZuce, Inc.

300 Brickstone Square

Suite 104

Andover, MA. 01810

O.978-296-1005 X2015 
M.207-956-0262
@mpicher <http://twitter.com/mpicher> 

linkedin
www.ezuce.com

Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee(s) named above. Any dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you. FMS

--

You received this message because you are subscribed to the Google Groups "sipxcom-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sipxcom-user...@googlegroups.com.
To post to this group, send email to sipxco...@googlegroups.com.
Visit this group at http://groups.google.com/group/sipxcom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/sipxcom-users/d2d9ad98-f189-4c38-b5fe-5a0eb9c64748%40googlegroups.com.

[Roger Northrop]

Looks like it is enabled.  I had a few more 408 and 480 errors today, and no slow Read Alert with them, so now am am really lost as to why calls fail.  1800 calls, and 28 failed, I know that is only.015% of the 1800, but I would think that with all of the memory, and processing power I have on the system would handle all calls at 100% completion rate.  is there any other adjustments that I can make.  I have sniffer traces of all of the calls, and it shows that it retries 4 times, then I get the error.  strange.. I will continue to look

CONFIG_NUMA=y

CONFIG_AMD_NUMA=y

CONFIG_X86_64_ACPI_NUMA=y

CONFIG_ACPI_NUMA=y

[Michael Picher]

Again, you want to *disable* NUMA...  you didn't say that you did that, just to be clear.

If you have SIP Capture enabled, disable it (there are 3 services).

I really don't think it's a server capacity issue...  1800 calls is nothing.  Specially over an entire day.

Back to basics...

Make sure you only have one NIC in the system.

Make sure your servers are pointing to themselves for DNS (check resolv.conf and also check the interface config files...).

I'll invoke Mike's theorem...  95% of the time, the problem is DNS.

Mike

Michael Picher, VP of Product Innovation
eZuce, Inc.

300 Brickstone Square

Suite 104

Andover, MA. 01810

O.978-296-1005 X2015 
M.207-956-0262
@mpicher <http://twitter.com/mpicher> 

linkedin
www.ezuce.com

Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee(s) named above. Any dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you. FMS

To view this discussion on the web visit https://groups.google.com/d/msgid/sipxcom-users/a72533df-0a9a-403e-bd0c-2c5110069d60%40googlegroups.com.

[Roger Northrop]

1. Disabled all 3 Sip Capture Services
2. NUMA...  ran the commands to see if NUMA is running but not getting any results 
2. # numactl --show
   bash: numactl: command not found
3. # numactl --hardware
   bash: numactl: command not found
5. # grep -i numa /var/log/dmesg
   NUMA: Allocated memnodemap from 17000 - 1b840
   NUMA: Using 20 for the hash shift.

I really do not know if Numa is really running.  I spoke to Data center team and they said that if it is running, they will not turn it off in BIOS

• I have one NIC
• DNS is itself

[Roger Northrop]

[PARTIALLY SOLVED]

1. Re-Enabled SIP Capture (I use this a lot so I really wanted to have it on)
2. Took lots of traces on my Oracle 1100 SBC
1. Found:  Up-stream SBC Sending a "CANCEL" right after an INVITE, then the 480 error would show up
1. this is telling me that it is not the SiPXCom system
3. I am still getting very few Mongo Slow Read Alerts 145ms - 800ms
1. Setting Changed:  in the Database settings, I changed the time out for Read Query from 100 to 1000
4. I Monitored the Dasboard of the SBC, and found that I was averaging as many as 42 sessions per second sustained during Busy hour
1. Setting Changed:  I went into the Media Services Settings and changed the Max Sessions per second from 30 - 50

So far today we have received 800 calls, and only 1, call was dropped due to a 480 error that the SipXcom sent and this error I am not able to track as to why this happend.

Conclusion:

• The reason why I feel that this is Partially solved, is because I caught the "CANCEL" coming from the upstream SBC and not from the SipXcom
• I can live with the occasional 480 error like today with 1, 480 error from the SipXCom out of 800 calls 

Right now the system is running much more smoothly than it has for the past 2 weeks.  Hope that this helps.