Proxy not reacting to branch response

12 views
Skip to first unread message

Raymond Domingo

unread,
Aug 5, 2009, 11:31:57 AM8/5/09
to mobicents-public
Intro:
When I recieve an invite I will first try to proxy it to host1, if I
don't recieve a response withing some time this response should be
cancelled and the invite should be sended to host2. As I understood
this is exactly what a sequencial proxy is all about.

Problem description:
When host1 doesn't respond to the invite it is indeed send to the
second host like supposed to.
BUT when host1 responses to the invite:
- the cancel is still sended to host1
- the response from host1 isn't proxied to my client
- the invite send to host2

Expected:
I would expect when host1 sends a response this response would be
proxied to my client and no further invites should go to hosts2.


My Code:
protected void doInvite(SipServletRequest request) throws
ServletException,
IOException {
logger.info
(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>");
logger.info("doInvite [" + request.getCallId() + "]");
logger.info
("--------------------------------------------------");

// initialize sipFactory
SipFactory sipFactory = (SipFactory) getServletContext
().getAttribute(
SIP_FACTORY);

// create proxy uri list
ArrayList<URI> proxyToList = new ArrayList<URI>();
String[] routingHosts = new String[] { "sip:
+XXXXXXXX@intranet1",
"sip:+XXX...@voip-test1.test.lan" };
for (String routingHost : routingHosts) {
URI destURI = sipFactory.createURI(routingHost);
proxyToList.add(destURI);
}


// the proxy
Proxy proxy = request.getProxy();

// try routinghost for 5 seconds and then advance to next
proxy.setProxyTimeout(5);

// set to true to receive bye
proxy.setRecordRoute(true);

// proxy sequentially / try routing hosts one by one
proxy.setParallel(false);

// proxy the call
proxy.proxyTo(proxyToList);

logger.info
("<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<");
}

Vladimir Ralev

unread,
Aug 5, 2009, 11:38:46 AM8/5/09
to mobicent...@googlegroups.com
Do you mean that after the first invite times out on the host1 and the
second invite goes to host2, host1 decides to send a response and then
the rest of the sequence occurs?

Raymond Domingo

unread,
Aug 5, 2009, 1:30:30 PM8/5/09
to mobicents-public
No, the invite doesnt time out because i get ringing from host1 . But
it seems proxy doesnt detect the response from host 1 because it also
sends the invite to host 2.

or is this feature ?

The goal i like to achieve is failover... If host1 is down/ doesnt
respont, host2 should be used. But when ringing is recieved from host
1, host2 should NOT be tried.
> >                "sip:+XXXXX...@voip-test1.test.lan" };

Vladimir Ralev

unread,
Aug 5, 2009, 1:34:56 PM8/5/09
to mobicent...@googlegroups.com
OK, I see, I will check the 289 spec to see what is the correct
behaviour but iirc if the client doesn't send final response (>=200)
we are supposed to start trying other phones. I will try to check
today.

Raymond Domingo

unread,
Aug 5, 2009, 2:23:58 PM8/5/09
to mobicents-public

The story continues... (a little more info) When my sipphone
responses ok to rining
of host 1 all gets instable because .... Invite is already proxied to
host 2

Vladimir Ralev

unread,
Aug 5, 2009, 6:48:30 PM8/5/09
to mobicent...@googlegroups.com
OK. This particular instability is probably caused by a bug in our
proxy. Our proxy should have canceled the first INVITE. I am fixing it
now. And I think I can see a problem in final response handling that
may account for the continues proxying. Tomorrow you should be able to
take a snapshot of MSS with these fixes. Thanks for the report.

Raymond Domingo

unread,
Aug 6, 2009, 3:23:26 AM8/6/09
to mobicents-public
Hi Vladimir,

First I would like to say thnx for looking in to my problem...

I just tried the new version, but I'm not sure if it got better....
(it didn't get worse!)

Remaining problem 1 (there are two):
------------------------------------------------
The (still?) remaining problem is that the ok message (when picking up
my callee phone) wasn't forwarded to the caller. So my caller's phone
is unaware that the callee picked up his phone...

Suggestion:
I continues looking in to it and it seems following change helps to
solve this problem (ProxyImpl.java:593) :
// Check if we are waiting for more response
if (!parallel || allResponsesHaveArrived()) { // Raymond: If
in SEQUENCIAL mode act on (first) final response
finalBranchForSubsequentRequests = bestBranch;
sendFinalResponse(bestResponse, bestBranch);
} else if (!parallel) {
startNextUntriedBranch();
}

In parallel mode all a final response can only be send when all
responses have arrived, but in sequencial mode you should
send the final response on the first final response received ?

After I changed this all seems to go fine !

Is this of any help to you ?

Remaining problem 2:
----------------------------
After all seems to go fine, I took a closer look at the network
traffic using wireshark. I see:
- cancel is sended to cancelled branches (seems ok)
- BUT also INVITE messages are stil send to cancelled branches. This
seems a bit strange, isn't it ?

Any suggestions ?

On 6 aug, 00:48, Vladimir Ralev <vladimir.ra...@gmail.com> wrote:
> OK. This particular instability is probably caused by a bug in our
> proxy. Our proxy should have canceled the first INVITE. I am fixing it
> now. And I think I can see a problem in final response handling that
> may account for the continues proxying. Tomorrow you should be able to
> take a snapshot of MSS with these fixes. Thanks for the report.
>

Vladimir Ralev

unread,
Aug 6, 2009, 3:36:11 AM8/6/09
to mobicent...@googlegroups.com
Yes, actually I was still investigating the problem, writing a
testcase. What I checked in earlier was not tested at all, but seemed
fine to me. This is the issue is created
http://code.google.com/p/mobicents/issues/detail?id=867 . You can
track it there. Thanks very much for you hints. I will definitely run
them with against my testcase and get back to you.

BTW, I think my patch should have solved your problem number 2, thus I
think you don't really have the latest binary. I noticed our hudson
was down earlier and it is likely this is the case. Since you already
know how to build from src code you can update. I added a cancel
within the lines you pasted that should solve problem 2 (untested as I
said before).

Raymond Domingo

unread,
Aug 6, 2009, 3:39:12 AM8/6/09
to mobicents-public
ok, I will test it right away

On 6 aug, 09:36, Vladimir Ralev <vladimir.ra...@gmail.com> wrote:
> Yes, actually I was still investigating the problem, writing a
> testcase. What I checked in earlier was not tested at all, but seemed
> fine to me. This is the issue is createdhttp://code.google.com/p/mobicents/issues/detail?id=867. You can
> track it there. Thanks very much for you hints. I will definitely run
> them with against my testcase and get back to you.
>
> BTW, I think my patch should have solved your problem number 2, thus I
> think you don't really have the latest binary. I noticed our hudson
> was down earlier and it is likely this is the case. Since you already
> know how to build from src code you can update. I added a cancel
> within the lines you pasted that should solve problem 2 (untested as I
> said before).
>

Raymond Domingo

unread,
Aug 6, 2009, 3:46:50 AM8/6/09
to mobicents-public
ok, I updated (my version ProxyImpl.java 7344) and retested.
Seems invites are still sended after cancel is sended...
see the wireshark log in attachement...

2009/8/6 Raymond Domingo <ray...@domingo.nl>:
proxy.wireshark

Vladimir Ralev

unread,
Aug 6, 2009, 3:51:02 AM8/6/09
to mobicent...@googlegroups.com
Hmm, ok thanks. It will be working soon I promise.

Vladimir Ralev

unread,
Aug 6, 2009, 7:37:03 AM8/6/09
to mobicent...@googlegroups.com
OK. It seems sequential proxying is very poorly covered by our tests
and the 289 TCK. I think they should be working for you now with the
code from trunk. Let me know if you still have problems. Thanks a lot
for the report and the analysis. Feel free to open issues directly in
our tracker.

Raymond Domingo

unread,
Aug 6, 2009, 8:09:45 AM8/6/09
to mobicent...@googlegroups.com
It's a better now.
But there is still an invite sent after a branch is cancelled, see
attached wireshark log.

You see
1 - invite is send to (dummy host) 74.125.77.121
2 - cancel is send to (dummy host) 74.125.77.121

3 - invite is send to (good host) 192.168.23.110
4 - (good host) 192.168.23.110 responses (trying, ringing, ok,....)

5 - an other invite is send to (dummy host) 74.125.77.121

Any idea what causes this ?

ps: It's my last day for the holliday. So if I don't reply, I will be
back 17 aug.

2009/8/6 Vladimir Ralev <vladimi...@gmail.com>:
proxy.wireshark

Vladimir Ralev

unread,
Aug 6, 2009, 10:55:14 AM8/6/09
to mobicent...@googlegroups.com
It is very hard to follow your wireshark, because there seem tobe many
retransmissions and phones do not OK the CANCEL and it's hard to tell
what the OK responses are for. Can you send a full wireshark (the
actual packets) ?

Raymond Domingo

unread,
Aug 6, 2009, 11:28:32 AM8/6/09
to mobicent...@googlegroups.com
This should be it.

2009/8/6 Vladimir Ralev <vladimi...@gmail.com>:
proxy_full.wireshark

Vladimir Ralev

unread,
Aug 6, 2009, 11:33:50 AM8/6/09
to mobicent...@googlegroups.com
Not quite. When you capture it packet by packet in wireshark you
should be able to do File -> Save in the so called pcap format that
can be opened from wireshark directly. What I need are the headers of
the sip message.

Raymond Domingo

unread,
Aug 6, 2009, 11:39:06 AM8/6/09
to mobicent...@googlegroups.com
retry :)
is this what you are looking for ?

2009/8/6 Vladimir Ralev <vladimi...@gmail.com>:

Vladimir Ralev

unread,
Aug 6, 2009, 11:44:25 AM8/6/09
to mobicent...@googlegroups.com
I dont see anything attached in this email :)

Raymond Domingo

unread,
Aug 7, 2009, 2:10:54 PM8/7/09
to mobicent...@googlegroups.com
sorry, something went wrong.
here it is, thnx in advance

talk to you after my holliday.

2009/8/6 Vladimir Ralev <vladimi...@gmail.com>:
proxy_full.wireshark

Vladimir Ralev

unread,
Aug 7, 2009, 2:16:16 PM8/7/09
to mobicent...@googlegroups.com
OK, this is what i wanted. Thanks. I will check it out.

Raymond Domingo

unread,
Aug 17, 2009, 10:21:55 AM8/17/09
to mobicents-public
This might be related....

During my stress tests (goes ok for +/- 20 minutes) then exception
below sometimes occurs. But I need to restart tomcat to solve the
problem :(

java.lang.IllegalStateException: Timer already cancelled.
at java.util.Timer.sched(Timer.java:354)
at java.util.Timer.schedule(Timer.java:170)
at org.mobicents.servlet.sip.proxy.ProxyBranchImpl.updateTimer
(ProxyBranchImpl.java:587)
at org.mobicents.servlet.sip.proxy.ProxyBranchImpl.start
(ProxyBranchImpl.java:256)
at org.mobicents.servlet.sip.proxy.ProxyImpl.startNextUntriedBranch
(ProxyImpl.java:555)
at org.mobicents.servlet.sip.proxy.ProxyImpl.startProxy
(ProxyImpl.java:446)
at org.mobicents.servlet.sip.proxy.ProxyImpl.proxyTo(ProxyImpl.java:
308)
at
nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.setupCall
(ForwardSipServlet.java:830)
at nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.doInvite
(ForwardSipServlet.java:263)
at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:225)
at
nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.doRequest
(ForwardSipServlet.java:132)
at javax.servlet.sip.SipServlet.service(SipServlet.java:325)
at
org.mobicents.servlet.sip.core.dispatchers.MessageDispatcher.callServlet
(MessageDispatcher.java:243)
at org.mobicents.servlet.sip.core.dispatchers.InitialRequestDispatcher
$1.dispatch(InitialRequestDispatcher.java:424)
at
org.mobicents.servlet.sip.core.dispatchers.DispatchTask.dispatchAndHandleExceptions
(DispatchTask.java:55)
at org.mobicents.servlet.sip.core.dispatchers.DispatchTask.run
(DispatchTask.java:50)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:651)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:676)
at java.lang.Thread.run(Thread.java:595)

On 7 aug, 20:16, Vladimir Ralev <vladimir.ra...@gmail.com> wrote:
> OK, this is what i wanted. Thanks. I will check it out.
>
> On Fri, Aug 7, 2009 at 9:10 PM, Raymond Domingo<raym...@domingo.nl> wrote:
> > sorry, something went wrong.
> > here it is, thnx in advance
>
> > talk to you after my holliday.
>
> > 2009/8/6 Vladimir Ralev <vladimir.ra...@gmail.com>:
>
> >> I dont see anything attached in this email :)
>
> >> On Thu, Aug 6, 2009 at 6:39 PM, Raymond Domingo<raym...@domingo.nl> wrote:
>
> >>> retry :)
> >>> is this what you are looking for ?
>
> >>> 2009/8/6 Vladimir Ralev <vladimir.ra...@gmail.com>:
>
> >>>> Not quite. When you capture it packet by packet in wireshark you
> >>>> should be able to do File -> Save in the so called pcap format that
> >>>> can be opened from wireshark directly. What I need are the headers of
> >>>> the sip message.
>
> >>>> On Thu, Aug 6, 2009 at 6:28 PM, Raymond Domingo<raym...@domingo.nl> wrote:
> >>>>> This should be it.
>
> >>>>> 2009/8/6 Vladimir Ralev <vladimir.ra...@gmail.com>:
>
> >>>>>> It is very hard to follow your wireshark, because there seem tobe many
> >>>>>> retransmissions and phones do not OK the CANCEL and it's hard to tell
> >>>>>> what the OK responses are for. Can you send a full wireshark (the
> >>>>>> actual packets) ?
> >>>>>> On Thu, Aug 6, 2009 at 3:09 PM, Raymond Domingo<raym...@domingo.nl> wrote:
> >>>>>>> It's a better now.
> >>>>>>> But there is still an invite sent after a branch is cancelled, see
> >>>>>>> attached wireshark log.
>
> >>>>>>> You see
> >>>>>>> 1 - invite is send to (dummy host) 74.125.77.121
> >>>>>>> 2 - cancel is send to (dummy host) 74.125.77.121
>
> >>>>>>> 3 - invite is send to (good host) 192.168.23.110
> >>>>>>> 4 - (good host) 192.168.23.110 responses (trying, ringing, ok,....)
>
> >>>>>>> 5 - an other invite is send to (dummy host) 74.125.77.121
>
> >>>>>>> Any idea what causes this ?
>
> >>>>>>> ps: It's my last day for the holliday. So if I don't reply, I will be
> >>>>>>> back 17 aug.
>
> >>>>>>> 2009/8/6 Vladimir Ralev <vladimir.ra...@gmail.com>:
>
> >>>>>>>> OK. It seems sequential proxying is very poorly covered by our tests
> >>>>>>>> and the 289 TCK. I think they should be working for you now with the
> >>>>>>>> code from trunk. Let me know if you still have problems. Thanks a lot
> >>>>>>>> for the report and the analysis. Feel free to open issues directly in
> >>>>>>>> our tracker.
>
> >>>>>>>> On Thu, Aug 6, 2009 at 10:51 AM, Vladimir Ralev<vladimir.ra...@gmail.com> wrote:
> >>>>>>>>> Hmm, ok thanks. It will be working soon I promise.
>
> >>>>>>>>> On Thu, Aug 6, 2009 at 10:46 AM, Raymond Domingo<raym...@domingo.nl> wrote:
> >>>>>>>>>> ok, I updated (my version ProxyImpl.java 7344) and retested.
> >>>>>>>>>> Seems invites are still sended after cancel is sended...
> >>>>>>>>>> see the wireshark log in attachement...
>
> >>>>>>>>>> 2009/8/6 Raymond Domingo <raym...@domingo.nl>:
> ...
>
> meer lezen »

Vladimir Ralev

unread,
Aug 18, 2009, 1:38:47 AM8/18/09
to mobicent...@googlegroups.com
Yes, this was reported by our QE lab and it is caused by leaked
timers. I would guess in your case it is caused by the non-existent
recipient and I am checking it again.

Raymond Domingo

unread,
Aug 18, 2009, 3:42:26 AM8/18/09
to mobicent...@googlegroups.com
I don't know if this should go into a new thread, but anyway I will
send you my last findings
I just updated trunk and rebuild sip-servlet-impl....

The IlegalStateException still seems a MAJOR show stopper to us.

When I run my simple stress test:
Just 10 threads (doing: invite + wait 3sec + bye, repeat)
Within one minute I already recieve the IlegalStateException.
And the bad thing is, tomcat/mobicents doesn't recover (at least not
within a limited amout of time and calls get lost)
I need to restart the tomcat in order to get it working again :(

Will you have the time to look into this ?

Your note "I would guess in your case it is caused by the non-existent
recipient" doesn't make any sense to me, because I run same test over
and over again. Most of the time it succeeds so it seems the recipient
exists.... ?

java.lang.IllegalStateException: Timer already cancelled.
at java.util.Timer.sched(Timer.java:354)
at java.util.Timer.schedule(Timer.java:170)
at org.mobicents.servlet.sip.proxy.ProxyBranchImpl.updateTimer(ProxyBranchImpl.java:587)
at org.mobicents.servlet.sip.proxy.ProxyBranchImpl.start(ProxyBranchImpl.java:256)
at org.mobicents.servlet.sip.proxy.ProxyImpl.startNextUntriedBranch(ProxyImpl.java:555)
at org.mobicents.servlet.sip.proxy.ProxyImpl.startProxy(ProxyImpl.java:446)
at org.mobicents.servlet.sip.proxy.ProxyImpl.proxyTo(ProxyImpl.java:308)
at nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.setupCall(ForwardSipServlet.java:830)
at nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.doInvite(ForwardSipServlet.java:263)
at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:225)
at nl.telecats.kasteel.fastvoip.sipservlets.ForwardSipServlet.doRequest(ForwardSipServlet.java:132)
at javax.servlet.sip.SipServlet.service(SipServlet.java:325)
at org.mobicents.servlet.sip.core.dispatchers.MessageDispatcher.callServlet(MessageDispatcher.java:243)
at org.mobicents.servlet.sip.core.dispatchers.InitialRequestDispatcher$1.dispatch(InitialRequestDispatcher.java:424)
at org.mobicents.servlet.sip.core.dispatchers.DispatchTask.dispatchAndHandleExceptions(DispatchTask.java:55)
at org.mobicents.servlet.sip.core.dispatchers.DispatchTask.run(DispatchTask.java:50)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
at java.lang.Thread.run(Thread.java:595)



2009/8/18 Vladimir Ralev <vladimi...@gmail.com>:

Vladimir Ralev

unread,
Aug 18, 2009, 3:44:11 AM8/18/09
to mobicent...@googlegroups.com
Ouch, please see which revision you have and post again.

Vladimir Ralev

unread,
Aug 18, 2009, 4:35:10 AM8/18/09
to mobicent...@googlegroups.com
Can you tell me what kind of timeout values you use?

Raymond Domingo

unread,
Aug 18, 2009, 4:51:47 AM8/18/09
to mobicent...@googlegroups.com
initial timeout (to wait for ringing 5sec)
after ringing received I set it to 180sec

2009/8/18 Vladimir Ralev <vladimi...@gmail.com>:
Reply all
Reply to author
Forward
0 new messages