Diameter Performance Issue

[Andrew Miller]

Folks,

I now have my Diameter application working.

It is very straight forward - It receives a Diameter Base accounting ACR
message, INSERTs a row into a database using JDBC and sends a ACA based
on the success of the INSERT.

It works fine, overnight I hit it with 1000 ACRs a second, resulting in
~ 58 Million INSERTS. Almost all were successfully.

However, in < 7000 instances I saw the following exception:

2012-01-11 02:59:07,549 ERROR
[javax.slee.RAEntityNotification[entity=DiameterBaseResourceAdaptor].DiameterBaseResourceAdaptor]
Error firing event.
javax.slee.resource.ActivityIsEndingException:
RA:DiameterBaseResourceAdaptor:Diameter Session
ID[aaa://192.168.0.36;178106...@192.168.0.212]
at
org.mobicents.slee.runtime.activity.ActivityContextImpl.fireEvent(ActivityContextImpl.java:517)
at
org.mobicents.slee.resource.SleeEndpointImpl._fireEvent(SleeEndpointImpl.java:770)
at
org.mobicents.slee.resource.SleeEndpointFireEventNotTransactedExecutor.execute(SleeEndpointFireEventNotTransactedExecutor.java:79)
at
org.mobicents.slee.resource.SleeEndpointImpl.fireEvent(SleeEndpointImpl.java:556)
at
org.mobicents.slee.resource.SleeEndpointImpl.fireEvent(SleeEndpointImpl.java:511)
at
org.mobicents.slee.resource.diameter.base.DiameterBaseResourceAdaptor.fireEvent(DiameterBaseResourceAdaptor.java:672)
at
org.mobicents.slee.resource.diameter.base.DiameterBaseResourceAdaptor.fireEvent(DiameterBaseResourceAdaptor.java:693)
at
org.mobicents.slee.resource.diameter.base.handlers.AccountingSessionFactory.doFireEvent(AccountingSessionFactory.java:172)
at
org.mobicents.slee.resource.diameter.base.handlers.AccountingSessionFactory.doAccRequestEvent(AccountingSessionFactory.java:121)
at
org.jdiameter.server.impl.app.acc.ServerAccSessionImpl.handleEventForStatelessMode(ServerAccSessionImpl.java:206)
at
org.jdiameter.server.impl.app.acc.ServerAccSessionImpl.handleEvent(ServerAccSessionImpl.java:142)
at
org.jdiameter.server.impl.app.acc.ServerAccSessionImpl.processRequest(ServerAccSessionImpl.java:443)
at
org.mobicents.slee.resource.diameter.base.DiameterBaseResourceAdaptor.processRequest(DiameterBaseResourceAdaptor.java:917)
at
org.mobicents.diameter.stack.DiameterStackMultiplexer.processRequest(DiameterStackMultiplexer.java:240)
at
org.jdiameter.server.impl.PeerImpl$LocalActionConext.consumeMessage(PeerImpl.java:571)
at
org.jdiameter.server.impl.PeerImpl$LocalActionConext.receiveMessage(PeerImpl.java:440)
at
org.jdiameter.server.impl.fsm.PeerFSMImpl$1.processEvent(PeerFSMImpl.java:121)
at
org.jdiameter.client.impl.fsm.PeerFSMImpl$3.run(PeerFSMImpl.java:161)
at java.lang.Thread.run(Thread.java:722)

I suspect that something is timing out and marking the activity as ended
before I get to process the ACR event.

I can see two ways forward:
1. Find out how the activity is being marked ended and work out how this
can be prevented or
2. Tune the performance of the SLEE to improve the performance or
increase the time-outs and see if the issue goes away.

Attempts to add logging have proved problematic, as any extra logging
makes things fall apart in more complicated ways !

I'm currently using 2.5.0 FINAL and I'm going to wait for 2.6.0 FINAL
and try it again before moving further. In the meantime I'd appreciate
any ideas on how to approach this from either direction and what bit of
code I should be looking at.

Andy Miller
Crocodile RCS Ltd.

[Alexandre Mendonça]

Hi Andy,

Have you had the chance to test with 2.6.0.FINAL ? Any better results ?

Regards,

--

Alexandre Mendonça // JBoss R&D
http://ammendonca.blogspot.com/

On Wed, Jan 11, 2012 at 16:21, Andrew Miller <andrew...@crocodile-rcs.com> wrote:

Folks,

I now have my Diameter application working.

It is very straight forward - It receives a Diameter Base accounting ACR message, INSERTs a row into a database using JDBC and sends a ACA based on the success of the INSERT.

It works fine, overnight I hit it with 1000 ACRs a second, resulting in ~ 58 Million INSERTS. Almost all were successfully.

However, in < 7000 instances I saw the following exception:

2012-01-11 02:59:07,549 ERROR [javax.slee.RAEntityNotification[entity=DiameterBaseResourceAdaptor].DiameterBaseResourceAdaptor] Error firing event.

javax.slee.resource.ActivityIsEndingException: RA:DiameterBaseResourceAdaptor:Diameter Session ID[aaa://192.168.0.36;17810689-8...@192.168.0.212]